Special procedure to upgrade Dynamic Engine to 1.3.0

Dynamic Engine 1.3.0 changes how services are installed. If you currently run Dynamic Engine version 1.2.x or earlier, you must prepare your cluster for upgrade by cleaning up or reassigning existing resources, regardless of whether you deployed using Helm or manifest files.

Attention: Follow this preparation procedure only if you are upgrading to Dynamic Engine 1.3.0 from a version earlier than 1.3.0. If you are installing Dynamic Engine 1.3.0 for the first time, these steps do not apply.

About this task

Dynamic Engine 1.3.0 introduces a significant architectural change in how Helm manages service charts. In earlier versions, Dynamic Engine used asynchronous installation, with services deployed as separate groups. Version 1.3.0 uses synchronous installation instead, embedding all services in the main Dynamic Engine and environment Helm charts.

This architectural shift requires two key preparation steps: reassigning resource ownership to the new structure and cleaning up legacy Helm metadata.

Procedure

Collect the identifiers for your Dynamic Engine and environments.
Before starting, gather the following information:
- Your Dynamic Engine ID (example: c-m-bdjp7fov) and its Kubernetes namespace (example: dynamic-engine-ns). Run this command to find them: kubectl get de -A
- Your Dynamic Engine environment ID (example: 6980a2850c0ed65c5a214d07) and its Kubernetes namespace (example: dynamic-engine-poc-env-ns). Run this command to find them: kubectl get dee -A
  Repeat this step for each environment you manage.
- Helm release names for your Dynamic Engine and environment: they default to dynamic-engine and dynamic-engine-env, installed in the default namespace.

Reassign core services to the new Dynamic Engine release.

The get command lists all Kubernetes resources belonging to the reloader and docker-registry services; the annotate command reassigns the ownership of these resources to the new consolidated Dynamic Engine release:

# check
kubectl get svc,secret,cm,pvc,deploy,cronjob,role,rolebinding,hpa,PodDisruptionBudget,NetworkPolicy,sa,clusterrole,clusterrolebinding \
  -n $DYNAMIC_ENGINE_NAMESPACE \
  -l 'app in (reloader, docker-registry)'

# patch
kubectl annotate svc,secret,cm,pvc,deploy,cronjob,role,rolebinding,hpa,PodDisruptionBudget,NetworkPolicy,sa \
  -n $DYNAMIC_ENGINE_NAMESPACE \
  -l 'app in (reloader, docker-registry)' \
  meta.helm.sh/release-name=dynamic-engine meta.helm.sh/release-namespace=default --overwrite

Replace the placeholder $DYNAMIC_ENGINE_NAMESPACE with your Dynamic Engine namespace, dynamic-engine-ns in this example.

Reassign environment services to the new environment release.

This get command lists all services in a given Dynamic Engine environment; the annotate command transfers all resources in your environment to the new consolidated environment release:

kubectl get svc,secret,cm,pvc,deploy,cronjob,role,rolebinding,hpa,PodDisruptionBudget,NetworkPolicy,sa,clusterrole,clusterrolebinding \
  -n $DYNAMIC_ENGINE_ENVIRONMENT_NAMESPACE \
  -l 'app.qlik.com/processing-environment-id=$DYNAMIC_ENGINE_ENVIRONMENT_ID'

kubectl annotate svc,secret,cm,pvc,deploy,cronjob,role,rolebinding,hpa,PodDisruptionBudget,NetworkPolicy,sa,clusterrole,clusterrolebinding \
  -n $DYNAMIC_ENGINE_ENVIRONMENT_NAMESPACE \
  -l 'app.qlik.com/processing-environment-id=$DYNAMIC_ENGINE_ENVIRONMENT_ID' \
  meta.helm.sh/release-name=dynamic-engine-env meta.helm.sh/release-namespace=default --overwrite

Replace the placeholders with your actual values, for example:

$DYNAMIC_ENGINE_ENVIRONMENT_NAMESPACE: dynamic-engine-poc-env-ns
$DYNAMIC_ENGINE_ENVIRONMENT_ID: 6980a2850c0ed65c5a214d07

Repeat this step once for each environment you have.

Remove some old Helm secrets from the Dynamic Engine deployment and its environments. Dynamic Engine v1.3.0 no longer uses these old secrets.

For Dynamic Engine:
```
kubectl get secret -n $DYNAMIC_ENGINE_NAMESPACE -l owner=helm,'name in (reloader, docker-registry)'

kubectl delete secret -n $DYNAMIC_ENGINE_NAMESPACE -l owner=helm,'name in (reloader, docker-registry)'
```
Replace the placeholder $DYNAMIC_ENGINE_NAMESPACE with your Dynamic Engine namespace.

This get command lists the secrets belonging to reloader and docker-registry; the delete command removes them.

For Dynamic Engine environments:

kubectl get secret -n $DYNAMIC_ENGINE_ENVIRONMENT_NAMESPACE \
 -l owner=helm,'name in (data-service-route-deployer-$DYNAMIC_ENGINE_ENVIRONMENT_ID, \
 di-job-controller-$DYNAMIC_ENGINE_ENVIRONMENT_ID,di-job-deployer-$DYNAMIC_ENGINE_ENVIRONMENT_ID, \
 engine-config-manager-$DYNAMIC_ENGINE_ENVIRONMENT_ID, log-collector-$DYNAMIC_ENGINE_ENVIRONMENT_ID)'

kubectl delete secret -n $DYNAMIC_ENGINE_ENVIRONMENT_NAMESPACE \
 -l owner=helm,'name in (data-service-route-deployer-$DYNAMIC_ENGINE_ENVIRONMENT_ID, \
 di-job-controller-$DYNAMIC_ENGINE_ENVIRONMENT_ID,di-job-deployer-$DYNAMIC_ENGINE_ENVIRONMENT_ID, \
 engine-config-manager-$DYNAMIC_ENGINE_ENVIRONMENT_ID, log-collector-$DYNAMIC_ENGINE_ENVIRONMENT_ID)'

Replace $DYNAMIC_ENGINE_ENVIRONMENT_NAMESPACE and $DYNAMIC_ENGINE_ENVIRONMENT_ID with the actual values.

Follow the standard upgrade procedure documented in Upgrading Dynamic Engine version to upgrade your Dynamic Engine and its environments to 1.3.0.
Step requirement: after downloading the deployment zip but before applying the values files, ensure the cloud region code property is present in the values file of each of your environments.
```
global:
  talendCloudRegion: <region_code>
```
You can find this region code in the URL of your Talend Management Console service.

Results

The upgrade succeeded and services are now consolidated. Run this command to verify:

helm ls -A

All services should show status deployed:

NAME                    NAMESPACE  STATUS    CHART
dynamic-engine-crd     default    deployed  dynamic-engine-crd-1.3.0
dynamic-engine         default    deployed  dynamic-engine-1.3.0
dynamic-engine-env     default    deployed  dynamic-engine-environment-1.3.0

All your data and configurations remain unchanged; only the way the services are organized has changed.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here