Implementing Dynamic Engine on a minikube for hands-on testing

Install and configure Dynamic Engines on a minikube cluster for hands-on testing before deploying them in your organization.

Prerequisites:

You must have Docker and minikube installed.
Ensure that the Docker daemon is running.
The minikube tool has been installed and configured to use a Kubernetes version supported by Dynamic Engines. For more information about the supported Kubernetes versions, see Kubernetes specifications.
For more information, see minikube installation.
The command-line tools kubectl and helm are installed. The Helm version must be 3.17.2 or later.

Adding a Dynamic Engine and its environment for testing on minikube

Procedure

In Talend Management Console, go to the Processing page, select the Dynamic Engines tab, and add a new Dynamic Engine.
This creates the logical entity of your Dynamic Engine. Close the deployment dialog without downloading — download the deployment zip file after assigning a Dynamic Engine environment.
Still in Talend Management Console, create a Dynamic Engine environment logical entity. For more information, see Adding a Dynamic Engine environment.
Return to the Processing page and open the Dynamic Engines tab.
Right-click the Dynamic Engine you created previously in Talend Management Console.
Click Assign environments to open the environment assignment page.
Select the environment you created and click Assign.
Keep the default Helm deployment type and version, then click Download to get the deployment zip file.
The zip file contains a Helm values file for the engine and one for each assigned environment, named after their respective IDs. For example, for a Dynamic Engine with ID c-m-sjufu4qy and one environment with ID 67f7d562ffd7c3525a902542, the zip contains:
- c-m-sjufu4qy-helm-values/c-m-sjufu4qy-values.yaml
- c-m-sjufu4qy-helm-values/67f7d562ffd7c3525a902542-values.yaml
Click Close.

Results

The Dynamic Engine and its environment are created as logical entities in Talend Management Console, the environment is assigned to the engine, and the deployment zip file is downloaded and ready to apply to your cluster.

Deploying Dynamic Engine and its environment on minikube

Procedure

Run the following command in your terminal to start a minikube instance:
```
minikube start -p QlikDE --cpus=4 --memory=8000
```
This command starts an instance named QlikDE and allocates 4 CPUs and 8 GB of memory. You can use any name for the instance.
Unzip the downloaded file and install the dynamic-engine-crd Helm chart:
```
helm install dynamic-engine-crd \
  oci://ghcr.io/talend/helm/dynamic-engine-crd \
  --version ${DYNAMIC_ENGINE_VERSION}
```
Replace <helm_chart_version> with the chart version supported by your Dynamic Engine version.

Deploy the engine instance:

helm install dynamic-engine-$DYNAMIC_ENGINE_ID \
  oci://ghcr.io/talend/helm/dynamic-engine \
  --version ${DYNAMIC_ENGINE_VERSION} -f $DYNAMIC_ENGINE_ID-values.yaml

Replace $DYNAMIC_ENGINE_ID with the Dynamic Engine ID, for example, c-m-sjufu4qy.

Verify that the engine was deployed successfully by running:

kubectl get de -A

You should see information similar to this:

NAME                                     VERSION   STATUS    LAST UPDATE
qlik-dynamic-engine-$DYNAMIC_ENGINE_ID   0.22.0    RUNNING   2m

Deploy the environment using Helm by running:
```
helm install dynamic-engine-environment-$DYNAMIC_ENGINE_ENVIRONMENT_ID \
  oci://ghcr.io/talend/helm/dynamic-engine-environment \
  --version ${DYNAMIC_ENGINE_VERSION} -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-values.yaml
```
Replace $DYNAMIC_ENGINE_ENVIRONMENT_ID with the Dynamic Engine environment ID, for example, 67f7d562ffd7c3525a902542.
The entire process takes 3 to 4 minutes depending on the capacity of your minikube machine. You can use the following commands to verify the progress:
- kubectl wait --for=condition=Ready pod -A -l 'app.qlik.com/part-of' --timeout=10m
  This command verifies that all environment-related pods are set up and running. Upon completion, a message like this is returned:
  pod/dynamic-engine-operator-68478645bb-wn7x9 condition met pod/dpe-di-job-deployer-86dc4ddb7f-7v9lz condition met pod/dpe-di-job-restore-orphans-6d4f8968d4-g75dk condition met pod/dpe-engine-config-manager-6bc9fb44-4z6nh condition met pod/dpe-job-controller-7c74467648-p7k8c condition met pod/dpe-log-collector-7fc4fb664d-scdf9 condition met
  Pods running in other namespaces are also listed in the message. For example, dynamic-engine-operator-68478645bb-wn7x9.
- kubectl get namespaces
  This command lists all the namespaces, allowing you to identify the namespace of the Dynamic Engine environment, which groups all associated services, such as qlik-processing-env-66f2c0bb3e063d3518122259.
- kubectl get pods --namespace qlik-processing-env-66f2c0bb3e063d3518122259
  This command lists all the services running in the environment. When all their statuses show Running, the installation of your Dynamic Engine environment is completed. In Talend Management Console, the status of this environment changes to Ready, indicating that it is ready to run tasks or plans.

Results

To run tasks in this Dynamic Engine environment, see Managing tasks in Dynamic Engine environments.

What to do next

To delete the minikube instance, run:

minikube delete --profile QlikDE

This removes the minikube instance and all its associated files.

For more information about Dynamic Engine logs, see Deactivating log transfer.
For known limitations of Dynamic Engines, see Known limitations.
For a complete guide about Dynamic Engines, see Dynamic Engine Configuration Guide. With this guide, you create and deploy Dynamic Engines and their environments for production.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here