Cloud-based Application Server and Shared Secret
When placing the Talend Data Catalog Application Server on the cloud, you will generally need to place remote harvesting servers on premises in order to harvest metadata locally that are not accessible from the cloud. Obviously, an organization would not want servers outside the organization’s on premises network to be able to invoke a harvesting server on premises. Thus, in the Over the Web (Cloud) mode, the harvesting is invoked according to a schedule on the harvesting server (not by the cloud-based Application Server) and the result is pushed up to the cloud.
Now, to ensure that no one may launch an attack on the Application Server in the cloud and pushing invalid metadata requests, both the application server and the on premises harvesting server use a shared secret between the cloud-based application server and the on premises harvesting server. In this scenario, both the URL in the UI and Server M_AGENT_SHARED_SECRET= in the agent.properties file of the on premises harvesting server must match before such a push of metadata is recognized or allowed.
Please refer to the installation instructions for more details and special instructions for the setup of the on premises harvesting server.
Example
- Specify the Type of on premises harvesting server to be Over the Web (Cloud).
- Enter a meaningful name in the Name field. Note, this name is what will appear when one is selecting the server to harvest from.
- Update the SHARED SECRET with the shared secret, in this case “SharedSecretLKDFLK*(L”.
- More details may be found in the deployment guide to update the agent.properties file of the on premises harvesting server with the same shared secret:
M_AGENT_SHARED_SECRET = SharedSecretLKDFLK*(L
As well as the URL of the application server.
Now the server is working.
The M_AGENT_SHARED_SECRET also acts as an identifier, so it must be made unique for a given application server. I.e., each remote harvesting agent must have a unique M_AGENT_SHARED_SECRET. The product does not use the agent name for anything except for display in the UI.