Deactivating logs and metrics transfer upon deployment or upgrade

During engine deployment or upgrade, prevent logs and metrics from being sent to Talend Management Console for tasks executed in a Dynamic Engine environment.

Note: This procedure only applies to deployments and upgrades of a Dynamic Engine environment. You can revert these settings with a subsequent upgrade.

If you need to deactivate the transfer of logs and metrics for deployed services, see Deactivating log and metrics transfers from a Dynamic Engine to Talend Management Console.

Before you begin

The dynamic-engine-crd custom resource definitions must have been installed using the oci://ghcr.io/talend/helm/dynamic-engine-crd helm chart. If not, run the following commands for the installation:
1. Find the chart version to be used:
  - Run the following Helm command:
    helm show chart oci://ghcr.io/talend/helm/dynamic-engine-crd --version <engine_version>
  - See the version directly from Talend Management Console or check the Dynamic Engine changelog for the chart version included in your Dynamic Engine version.
  - Use an API call to the Dynamic Engine version endpoint.
2. Run the following command to install the Helm chart of a given version:
```
helm install dynamic-engine-crd oci://ghcr.io/talend/helm/dynamic-engine-crd --version <helm_chart_version>
```
  Replace <helm_chart_version> with the chart version supported by your Dynamic Engine version.
  Without specifying the version, you install the latest available dynamic-engine-crd chart version.
Your Dynamic Engine must have been deployed. If not, run this command to deploy it:
```
helm install dynamic-engine -f <engine-id>-helm-values/<engine-id>-values.yaml oci://ghcr.io/talend/helm/dynamic-engine 
```
This is the default deployment, because the current customization is only needed on the Dynamic Engine environment side. For this reason no customization is configured for the Dynamic Engine instance.

About this task

By default, logs and metrics from tasks running in a Dynamic Engine environment are sent to Talend Management Console. You can deactivate this behavior at deployment or upgrade time by customizing the environment configuration.

Procedure

In the Kubernetes machine, unzip the Helm deployment zip file previously downloaded.

Create a custom values file to deactivate logs and metrics transfer from your Dynamic Engine environment.

cat <<EOF > $DYNAMIC_ENGINE_ENVIRONMENT_ID-custom-log-metrics-values.yaml
configuration:
  logging:
    publishToTalendCloud: false
  observability:
    componentMetricsFeatureEnabled: false
EOF

Deploy the Dynamic Engine environment with logging and metrics transfer deactivated.

helm install dynamic-engine-environment-$DYNAMIC_ENGINE_ENVIRONMENT_ID -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-values.yaml  \
 -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-custom-log-metrics-values.yaml \
 oci://ghcr.io/talend/helm/dynamic-engine-environment \
 --version $DYNAMIC_ENGINE_VERSION

Tip: When changes are needed, you can update the content of the $DYNAMIC_ENGINE_ENVIRONMENT_ID-custom-log-metrics-values.yaml file and run helm upgrade to apply the changes:

helm upgrade dynamic-engine-environment-$DYNAMIC_ENGINE_ENVIRONMENT_ID -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-values.yaml  \
-f $DYNAMIC_ENGINE_ENVIRONMENT_ID-custom-log-metrics-values.yaml \
oci://ghcr.io/talend/helm/dynamic-engine-environment \
--version $DYNAMIC_ENGINE_VERSION

Verify that logs and metrics transfer is deactivated in the environment configuration.

Check the logging configuration:

kubectl get configmap -n qlik-processing-env-$DYNAMIC_ENGINE_ENVIRONMENT_ID \
global-configuration -ojsonpath="{.data['global-configuration\.json']}" | jq -r '.logging'

Confirm that publishToTalendCloud is set to false.

Check the observability configuration:

kubectl get configmap -n qlik-processing-env-$DYNAMIC_ENGINE_ENVIRONMENT_ID \
global-configuration -ojsonpath="{.data['global-configuration\.json']}" | jq -r '.observability'

Confirm that componentMetricsFeatureEnabled is set to false.

Results

The Dynamic Engine environment is deployed with logs and metrics transfer deactivated. No execution logs or metrics will be sent to Talend Management Console for task runs in this environment until you re-activate this feature.

In Talend Management Console, the status of this environment becomes Ready, confirming that it is ready to run tasks or plans.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here