Deactivating log and metrics transfers from a Dynamic Engine to Talend Management Console
By default, execution logs and Job component metrics are automatically transferred to Talend Management Console and can be viewed on the Run overview page. You can deactivate this transfer.
Update the ConfigMap called global-configuration of the Dynamic Engine environment in Kubernetes to deactivate the log transfer from the associated Dynamic Engine.
For more information on kubectl commands and flags, see kubectl Quick Reference.
About this task
Understanding the deployment architecture of a Dynamic Engine instance and its Dynamic Engine environments in a Kubernetes cluster helps perform the updates successfully:
- Dynamic Engine and Dynamic Engine environments:
Each Dynamic Engine environment is assigned to only one Dynamic Engine at a time but multiple environments are allowed per engine.
-
Deployment restriction:
Only one Dynamic Engine is allowed in the Kubernetes cluster but multiple Dynamic Engine environments can be deployed.
-
Configuration isolation:
Each Dynamic Engine environment has its own dedicated global-configuration ConfigMap. Updates to a specific ConfigMap only affect the environment to which it belongs.
-
Custom resource type:
Dynamic Engine environments are deployed as a custom Kubernetes resource definition called QlikRuntime. The resource name of a Dynamic Engine environment is also used as its namespace name.
Procedure
Results
- This command returns the logs about the service that creates the Kubernetes processes.
kubectl logs --tail=20 -n qlik-processing-env-66f2c0bb3e063d3518122259 -l 'app.kubernetes.io/instance=di-job-controller'
The --tail option allows you to specify the number of lines of log output to display.
- This command returns the logs about the service that reads the task message from ActiveMQ and triggers the Kubernetes processes.
kubectl logs --tail=20 -n qlik-processing-env-66f2c0bb3e063d3518122259 -l 'app.kubernetes.io/instance=di-job-deployer'
- This command returns the logs of a given task. The 'flowID=664c66ddb1ad64071d44b510' field is the ID of the task in question. You can find this ID on the dedicated page of the task in Talend Management Console or by using this API endpoint.
kubectl logs --tail=20 -n qlik-processing-env-66f2c0bb3e063d3518122259 -l 'flowID=664c66ddb1ad64071d44b510'
- This command returns the logs of a given execution. The 'flowExecutionId=3c1d860a-e433-47c0-90bb-52a3b2b4fdfa' field is the ID of the task or plan execution in question. You can find this ID using this API endpoint or on the Task execution log page in Talend Management Console. On the Task execution log page, the ID is labeled as Task execution ID.
kubectl logs --tail=20 -n qlik-processing-env-66f2c0bb3e063d3518122259 -l 'flowExecutionId=3c1d860a-e433-47c0-90bb-52a3b2b4fdfa'
You can use -f to specify if the logs should be streamed.
You can also use kubectl logs --help to see all commands and options.