Deactivating logs and metrics transfer upon deployment or upgrade

During engine deployment or upgrade, prevent logs and metrics from being sent to Talend Management Console for tasks executed in a Dynamic Engine environment.

Note: This procedure only applies to deployments and upgrades of a Dynamic Engine environment. You can revert these settings with a subsequent upgrade.

If you need to deactivate the transfer of logs and metrics for deployed services, see Deactivating log and metrics transfers from a Dynamic Engine to Talend Management Console.

About this task

By default, logs and metrics from tasks running in a Dynamic Engine environment are sent to Talend Management Console. You can deactivate this behavior at deployment or upgrade time by customizing the environment configuration.

Procedure

Use the dynamic-engine-crd Helm chart to install the Dynamic Engine custom resource definitions of a given Helm version.
1. The chart version to use is the same as your Dynamic Engine version. You can find this version using one of the following methods:
  - Run the following Helm command:
    helm show chart oci://ghcr.io/talend/helm/dynamic-engine --version <engine_version>
  - See the version directly from Talend Management Console or check the Dynamic Engine changelog for the chart version included in your Dynamic Engine version.
  - Use an API call to the Dynamic Engine version endpoint.
2. In the Kubernetes cluster, run the following command to install the Helm chart of a given version:
```
helm install dynamic-engine-crd oci://ghcr.io/talend/helm/dynamic-engine-crd --version <helm_chart_version>
```
  Replace <helm_chart_version> with the chart version supported by your Dynamic Engine version.
  Without specifying the version, you install the latest available dynamic-engine-crd chart version.
Still in the Kubernetes machine, unzip the Helm deployment zip file previously downloaded.

Create a custom values file to deactivate logs and metrics transfer from your Dynamic Engine environment.

cat <<EOF > <environment-id>-custom-log-metrics-values.yaml
configuration:
  logging:
    publishToTalendCloud: false
  observability:
    componentMetricsFeatureEnabled: false
EOF

Run this command to deploy the Dynamic Engine instance:
```
helm install dynamic-engine -f <engine-id>-helm-values/<engine-id>-values.yaml oci://ghcr.io/talend/helm/dynamic-engine
```
This is the default deployment, because this deactivation is only needed on the Dynamic Engine environment side. For this reason no customization has been done for the Dynamic Engine instance.

Deploy the Dynamic Engine environment with logging and metrics transfer deactivated.

helm install dyn-engine-env-<environment-id> -f <environment-id>-values.yaml  \
 -f <environment-id>-custom-log-metrics-values.yaml \
 oci://ghcr.io/talend/helm/dynamic-engine-environment \
 --version <engine-version>

Tip: When changes are needed, you can update the content of the <environment-id>-custom-log-metrics-values.yaml file and run helm upgrade to apply the changes:

helm upgrade dyn-engine-env-<environment-id> -f <environment-id>-values.yaml  \
-f <environment-id>-custom-log-metrics-values.yaml \
oci://ghcr.io/talend/helm/dynamic-engine-environment \
--version <engine-version>

Verify that logs and metrics transfer is deactivated in the environment configuration.

Check the logging configuration:

kubectl get configmap -n qlik-processing-env-<environment-id> \
global-configuration -ojsonpath="{.data['global-configuration\.json']}" | jq -r '.logging'

Confirm that publishToTalendCloud is set to false.

Check the observability configuration:

kubectl get configmap -n qlik-processing-env-<environment-id> \
global-configuration -ojsonpath="{.data['global-configuration\.json']}" | jq -r '.observability'

Confirm that componentMetricsFeatureEnabled is set to false.

Results

The Dynamic Engine environment is deployed with logs and metrics transfer deactivated. No execution logs or metrics will be sent to Talend Management Console for task runs in this environment until you re-activate this feature.

In Talend Management Console, the status of this environment becomes Ready, confirming that it is ready to run tasks or plans.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here