Deactivating log and metrics transfers from a Dynamic Engine to Talend Management Console

By default, execution logs and Job component metrics are automatically transferred to Talend Management Console and can be viewed on the Run overview page. You can deactivate this transfer.

Update the ConfigMap called global-configuration of the Dynamic Engine environment in Kubernetes to deactivate the log transfer from the associated Dynamic Engine.

For more information on kubectl commands and flags, see kubectl Quick Reference.

About this task

Understanding the deployment architecture of a Dynamic Engine instance and its Dynamic Engine environments in a Kubernetes cluster helps perform the updates successfully:

Dynamic Engine and Dynamic Engine environments:
Each Dynamic Engine environment is assigned to only one Dynamic Engine at a time but multiple environments are allowed per engine.
Deployment restriction:

Only one Dynamic Engine is allowed in the Kubernetes cluster but multiple Dynamic Engine environments can be deployed.
Configuration isolation:

Each Dynamic Engine environment has its own dedicated global-configuration ConfigMap. Updates to a specific ConfigMap only affect the environment to which it belongs.
Custom resource type:

Dynamic Engine environments are deployed as a custom Kubernetes resource definition called QlikRuntime. The resource name of a Dynamic Engine environment is also used as its namespace name.

Procedure

Get the namespace name of the Dynamic Engine environment to be updated by running this command:
```
kubectl get qlikruntime
```
A list of resources of this QlikRuntime type is returned. This allows you to identify the Dynamic Engine environment for which you need to update its ConfigMap. For example, the resource name or the namespace name to be used can read like: qlik-processing-env-66f2c0bb3e063d3518122259.
Open the global-configuration ConfigMap to update the property about sending logs to Talend Management Console:
```
kubectl edit cm -n qlik-processing-env-66f2c0bb3e063d3518122259 global-configuration
```
In this command, cm represents ConfigMap, -n means namespace, and qlik-processing-env-66f2c0bb3e063d3518122259 is the example namespace name of the Dynamic Engine environment resource.
The global-configuration ConfigMap is opened in your text editor.
Change each of the values from true to false for the following properties to deactivate the transfers:
- "logging":{"publishToTalendCloud":false}: this property defines whether execution logs are transferred to Talend Management Console.
- "observability":{"componentMetricsFeatureEnabled": false}: this property determines whether execution metrics are transferred to Talend Management Console.
Save the file.

Results

After this change, the logs, although still created in the engine, are not sent to Talend Management Console.

In this situation, if you need to access the logs, run the following commands in your Kubernetes terminal:

```
kubectl logs --tail=20 -n qlik-processing-env-66f2c0bb3e063d3518122259 -l 'app.kubernetes.io/instance=di-job-controller'
```
This command returns the logs about the service that creates the Kubernetes processes.
The --tail option allows you to specify the number of lines of log output to display.
```
kubectl logs --tail=20 -n qlik-processing-env-66f2c0bb3e063d3518122259 -l 'app.kubernetes.io/instance=di-job-deployer'
```
This command returns the logs about the service that reads the task message from ActiveMQ and triggers the Kubernetes processes.
```
kubectl logs --tail=20 -n qlik-processing-env-66f2c0bb3e063d3518122259 -l 'flowID=664c66ddb1ad64071d44b510'
```
This command returns the logs of a given task. The 'flowID=664c66ddb1ad64071d44b510' field is the ID of the task in question. You can find this ID on the dedicated page of the task in Talend Management Console or by using this API endpoint.
```
kubectl logs --tail=20 -n qlik-processing-env-66f2c0bb3e063d3518122259 -l 'flowExecutionId=3c1d860a-e433-47c0-90bb-52a3b2b4fdfa'
```
This command returns the logs of a given execution. The 'flowExecutionId=3c1d860a-e433-47c0-90bb-52a3b2b4fdfa' field is the ID of the task or plan execution in question. You can find this ID using this API endpoint or on the Task execution log page in Talend Management Console. On the Task execution log page, the ID is labeled as Task execution ID.

Tip:

You can use -f to specify if the logs should be streamed.

You can also use kubectl logs --help to see all commands and options.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here