Remote Engine configuration files

Remote Engine generates its own key pair (RSA/4096 bits) during the pairing process. With Talend Remote Engine v2.12.5 onwards, this key pair is stored in a Java keystore in etc/keystores, secured with a password defined in a dedicated configuration file or as an environment variable.

If no password is defined, a random value is generated instead to ensure that the private key is still stored encrypted at rest.

This pair of keys is used to protect the information transferred between Talend Remote Engine and Talend Management Console.

• Activate or deactivate log transmission from Remote Engine to Talend Management Console. See Preventing the engines from sending logs to Talend Cloud.
• Send logs to a third-party system, such as a JMS appender, a Syslog appender or an external folder. See Setting up custom logging.
• Change engine log filters to include or exclude artifact execution logs. See Restoring execution log duplication.

• Add org.osgi.service.http.enabled = false to prevent the Remote Engine from listening on port 8043 (HTTP) and use the secure Remote Engine port instead (9004).

  For further information about how to access this port using trusted client certificate, see Creating trusted client certificate.

• Enable remote access to Remote Engine from other machines. See Remote access to Remote Engine.

• The local.maven.clean.interval.days parameter controls automatic cleanup of unused artifacts in your engine. For further information about your Remote Engine's resilience settings, see Automatic artifact cleanup.
• Set maximum number of parallel runs on a single engine or a engine cluster. For further information about parallel runs on a Remote Engine, see Running tasks in parallel.
• Set up a regular cleanup process to automatically delete unused artifacts. See

/etc/org.talend.ipaas.rt.dsrunner.cfg

• (Linux only) Configure impersonate users to run data service or Route tasks and activate the sudo feature. See Impersonate users for data service or Route tasks and Start Remote Engine with sudo.
• Set custom bundle deployment directory for Talend Runtime.

For further information about how to define your own password to encrypt the keystore, see Securing keystores to access Remote Engine.

See Display standard system logs in Talend Management Console.

• Add the following parameter to mask the locked connection parameters in the task run logs:

  job.log.msg.sensitive.hidden=true

  For further information, see Sensitive information in logs.

• Add the following parameters to mitigate CPU and memory consumption spike:
  • job.log.msg.size.max
  • job.log.msg.buffer.size.max
  • thread.timeout

  For further information, see the troubleshooting section in Sensitive information in logs.

• With Remote Engine v2.12.9 onwards, add the following parameter to encrypt the password type context parameters of a Job task:

  job.ctx.params.file.sensitive.values.encrypted=true

• Change the Talend Remote Engine Job log level. See Adjusting Remote Engine Job log levels.
• Adjust the log cleanup lifecycle. See Automatic task log cleanup.
• Use logging configuration from Talend Studio. See Using the custom logging configuration that is defined in Talend Studio.

• Add the configuration to be used to create connection between Talend Studio and Talend Remote Engine.

  For further information, see Configuring connection between Talend Studio and Remote Engine.

• Set up JobServer server configuration for an SSL connection between the JobServer server and the JobServer client inside a Remote Engine. The JobServer client configuration is added in etc/system.properties.

  For further information about SSL connection in a Remote Engine, see Activating SSL connection.

• (Linux only) Configure impersonate users to run data integration tasks and activate the sudo feature. See Impersonate users for data integration tasks and Start Remote Engine with sudo.
• Make Remote Engines use your own signing key to verify artifact signatures. See Remote Engines use a custom signing key.
• Set harder limits for folders and file names to mitigate or prevent Denial of Service attacks. See Improving security in case of malicious archive content.

For further information about how to pair your engine using this file, see Pairing Remote Engines with the configuration file.

• Activate graceful reboot. See Graceful reboot on Linux.
• Add the client configuration and the server configuration of your JobServer to separate files to establish an SSL connection between the JobServer server and the JobServer client inside a Remote Engine. The client configuration should be added to this /etc/system.properties file, while the server configuration should be added to /etc/org.talend.remote.jobserver.server.cfg.

  For further information about SSL connection in a Remote Engine, see Activating SSL connection.

• Configure proxy settings for a Remote Engine. See Proxy settings for Remote Engines and Proxy authentication.

In this file, if necessary, define the specific system environment properties to be used by the engine. For example:

• In the JVM parameters section of this wrapper file, specify the JVM settings to be used by the engine. Note that these JVM settings determine the resources to be allocated to the engine as a service, while the JVM settings defined in a Run profile in Talend Management Console refer to the resources allocated to that specific execution.
• If the Job to be run on a Remote Engine relies on third-party tools, such as an SDK, or MelissaData, specify the paths to these tools in this Remote Engine wrapper configuration file using the following format:

  set.PATH=<PATH_TO_SDK>;%PATH%

  This line informs the engine about the location to the third-party tool while retaining the existing content of the PATH environment variable. For further information, see Enabling the execution of Jobs using external tools (SDK).

For further information, see Managing Remote Engine users and passwords.

For further information about how to deploy multiple engines, see Deploying multiple engines.

/TalendJobServersFiles/jobexecutions/logs/