Using Helm with Dynamic Engine

Beta

Starting from Dynamic Engine v0.20.0 (R2025-03), you can set up a Dynamic Engine instance and its environments in Kubernetes using Dynamic Engine Helm charts.

Using Dynamic Engine Helm charts is an advanced approach to deploy Dynamic Engine instances.

Dynamic Engine Helm charts are available as images under oci://ghcr.io/talend/helm/ and include:

dynamic-engine-crd: Core custom resource definitions for Dynamic Engine services.
dynamic-engine: Helm chart for a Dynamic Engine instance.
dynamic-engine-environment: Helm chart for Dynamic Engine environments.

Before you begin

A Kubernetes(version 1.28 or later) cluster without existing Dynamic Engine services. Dynamic Engine Helm charts will be installed in this cluster.
Privileged access to this Kubernetes cluster.
Installed command-line tools: kubectl and helm. The Helm version must be 3.17.2 or later.
An existing Dynamic Engine instance with at least one assigned environment in Talend Management Console. To create a Dynamic Engine instance and assign environments, see Adding a Dynamic Engine.

Generating Helm values file for Dynamic Engine deployment

Beta

A Helm values file contains the configuration information such as IDs and bootstrap keys specific to your Dynamic Engine instance and its associated environments.

About this task

After setting up your Dynamic Engine instance with its assigned environments in Talend Management Console, you can generate the Helm values files using the dedicated API endpoint.

For example, assuming the Dynamic Engine ID is c-m-sjufu4qy and it has one associated environment. Use this API to generate an archive containing the Helm values files.

Procedure

In a command-line terminal, run the following curl command:
```
curl -H "Authorization: Bearer your_personal_access_token" \ -X POST \ https://api.eu.cloud.talend.com/processing/runtimes/dynamic-engines/c-m-sjufu4qy/actions/generate-helm \ -H "Content-Type: application/json" \ -d "{\"generateNewPairingKey\": true}"
```
Information noteTip: For the first deployment, setting the generateNewPairingKey option is not required. This option generates new pairing keys for all Dynamic Engine environments assigned to the Dynamic Engine instance.
Locate the generated zip file named c-m-sjufu4qy.zip and extract it.
This zip contains the following files:
- For Dynamic Engine: c-m-sjufu4qy-helm-values/c-m-sjufu4qy-values.yaml
- For Dynamic Engine environment: c-m-sjufu4qy-helm-values/67f7d562ffd7c3525a902542-values.yaml (One file for one distinct environment, with the environment ID in the file name).

Results

You will need to use these Helm values files later for Dynamic Engine deployment.

Installing Dynamic Engine custom resource definitions (CRDs) with Helm chart

Beta

Use the dynamic-engine-crd Helm chart to install the Dynamic Engine custom resource definitions.

Procedure

The chart version to use is the same as your Dynamic Engine version. You can find this version using one of the following methods:
- Run the following Helm command:
```
helm show chart oci://ghcr.io/talend/helm/dynamic-engine --version $DYN_ENGINE_VERSION
```
- See the version directly from Talend Management Console or check the Dynamic Engine changelog for the chart version included in your Dynamic Engine version.
- Use an API call to the Dynamic Engine version endpoint.
In the Kubernetes cluster, run the following command to install the Helm chart of a given version:
```
helm install dynamic-engine-crd oci://ghcr.io/talend/helm/dynamic-engine-crd --version <version>
```
Replace <version> with the chart version supported by your Dynamic Engine version.
Without specifying the version, you install the latest available dynamic-engine-crd chart version.

Installing Dynamic Engine and its associated environment

Beta

Use the dynamic-engine and dynamic-engine-environment Helm charts to deploy the Dynamic Engine instance and its environment.

About this task

As explained earlier, in this example, the Helm values files are:

For Dynamic Engine: c-m-sjufu4qy-helm-values/c-m-sjufu4qy-values.yaml
For Dynamic Engine environment: c-m-sjufu4qy-helm-values/67f7d562ffd7c3525a902542-values.yaml

Procedure

Run this command to deploy the engine instance:

helm install dynamic-engine -f c-m-sjufu4qy-helm-values/c-m-sjufu4qy-values.yaml oci://ghcr.io/talend/helm/dynamic-engine

Verify Dynamic Engine status:

Using Helm:
```
helm status dynamic-engine
```
Using kubectl:
```
kubectl get de
```

You should see information similar to this:

NAME                               VERSION   STATUS    LAST UPDATE
qlik-dynamic-engine-c-m-sjufu4qy   0.20.0    RUNNING   2m

Run this command to deploy the Dynamic Engine environment.

helm install dyn-engine-env -f c-m-sjufu4qy-helm-values/67f7d562ffd7c3525a902542-values.yaml oci://ghcr.io/talend/helm/dynamic-engine-environment

Verify environment resources.
1. Run the following command:
```
kubectl get dee
```
  You should see information reading like this:
```
NAME                                           VERSION   STATUS    LAST UPDATE
qlik-processing-env-67f7d562ffd7c3525a902542   0.20.0    RUNNING   <invalid>
```
2. In Talend Management Console, verify the Dynamic Engine environment status.
  The status should show as Active.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here