Resource settings for Talend Management Console task executions
Services in a Dynamic Engine environment deployed on your Kubernetes cluster execute Talend Management Console tasks by creating and managing Kubernetes jobs with the resource settings specified through run profiles.
For details about how to use a run profile with tasks, see Creating a run profile.
- Kubernetes jobs: in the namespace of a Dynamic Engine environment, each task execution is mapped to a Kubernetes job. The job-controller service running in this namespace creates these Kubernetes jobs.
- Deployments of Data Services and Routes: In the namespace of a Dynamic Engine environment, each Data Service or Route task is assigned to a Kubernetes deployment. The data-service-route-deployer service running in this namespace creates these Kubernetes deployments.
- CPU and memory settings:
- Default values for Jobs (including Data Services):
- CPU: Requests 100m (0.1), Limits 300m (0.3)
- Memory: Requests 256 MB, Limits 1 GB
-
Default values for Routes
-
CPU: Requests 100m (0.1), Limits 300m (0.3)
-
Memory: Requests 256 MB, Limits 1.2 GB
-
- Custom settings:
To use different settings, specify JVM parameters in each task's run profile. Dynamic Engine services calculate the CPU and memory requests and limits for the corresponding Kubernetes pods. For memory, the calculation uses a default ratio of 1.2 between the estimated maximum RAM for JVM and the pod RAM request, capping the maximum pod RAM request at 7 GB.
- Additional JVM settings:
You can also specify other JVM parameters for the pods, such as Java heap size and heap percentage.
- JVM settings application:
All the JVM settings are applied during the task execution time.
- Default values for Jobs (including Data Services):
-
Default Docker base image:
- For Talend Management Console Job tasks (including Data Service Jobs): Public base image from Qlik, based on OpenJDK 17 and OpenJDK JRE 17.
- For Talend Management Console Route tasks: public base image from Qlik, based on OpenJDK 17 and OpenJDK JRE 17.
Ensure to set up Java 17 in Studio to build your Jobs. For further information, see Setting up Java in Studio.