Deploying Dynamic Engine with minimal images

Reduce the attack surface and image size of your Dynamic Engine deployment by switching all Dynamic Engine services to distroless container images and Talend Management Console task images to minimal container images with a very light OS.

Setting global.profile: minimal in your Helm values file switches every Dynamic Engine service image to its distroless variant. Once done, infrastructure service images use the -distroless tag suffix; data integration tasks and Data Service or Route images use the -minimal tag suffix.

Distroless images contain no shell and no package manager, which reduces the filesystem footprint and limits the attack surface exposed to potential exploits.

This configuration is recommended for managed Kubernetes modes, including Amazon EKS Auto Mode and GKE Autopilot.

Support for distroless images was introduced in Dynamic Engine v1.6.0.

About this task

Important:

For managed Kubernetes modes (Amazon EKS Auto Mode and GKE Autopilot), use this configuration with PodDisruptionBudget configuration and configuration.persistence.enabled=false.

Important:

Big data Jobs (Spark Batch and Spark Streaming) are incompatible with minimal images.

When using global.profile: minimal, override each big data Job by adding the following processing argument to its corresponding run profile:

-K8S:ImageProfile=standard

This run profile parameter overrides global.profile at the individual task level, so the big data Job's container uses the standard full base image while other tasks use the minimal image. For information about creating and editing run profiles, see Managing Run profiles for Dynamic Engine environment tasks.

Procedure

Create a custom Helm values file that sets global.profile to minimal.
```
cat <<EOF > custom-minimal-image-values.yaml
---
global:
  profile: minimal
EOF
```
This single values file is applied to both the dynamic-engine and dynamic-engine-environment charts to ensure all services use distroless images.

Install or upgrade the dynamic-engine-crd, dynamic-engine, and dynamic-engine-environment charts, passing the custom values file to both runtime charts.

DYNAMIC_ENGINE_VERSION=1.6.0
DYNAMIC_ENGINE_ID=<engine_id>
DYNAMIC_ENGINE_ENVIRONMENT_ID=<environment_id>

helm upgrade --install dynamic-engine-crd \
  oci://ghcr.io/talend/helm/dynamic-engine-crd --version $DYNAMIC_ENGINE_VERSION

helm upgrade --install dynamic-engine-$DYNAMIC_ENGINE_ID \
  oci://ghcr.io/talend/helm/dynamic-engine --version $DYNAMIC_ENGINE_VERSION \
  -f $DYNAMIC_ENGINE_ID-values.yaml \
  -f custom-minimal-image-values.yaml

helm upgrade --install dynamic-engine-environment-$DYNAMIC_ENGINE_ENVIRONMENT_ID \
  oci://ghcr.io/talend/helm/dynamic-engine-environment --version $DYNAMIC_ENGINE_VERSION \
  -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-values.yaml \
  -f custom-minimal-image-values.yaml

Replace <engine_id> and <environment_id> with your actual engine and environment identifiers.

Results

All Dynamic Engine service pods restart and pull distroless image variants. You can verify the active profile with the following commands:

helm get values dynamic-engine-$DYNAMIC_ENGINE_ID --output json | jq -r '.global.profile'
helm get values dynamic-engine-environment-$DYNAMIC_ENGINE_ENVIRONMENT_ID --output json | jq -r '.global.profile'

Both commands should return minimal.

What to do next

If any of your tasks are big data Jobs, create or update the corresponding run profile to include -K8S:ImageProfile=standard. See Managing Run profiles for Dynamic Engine environment tasks.

Troubleshooting minimal image deployments

Diagnose common issues that occur after switching to distroless container images.

Procedure

If services continue using standard images after the upgrade, verify that the custom values file was applied to both charts.

The custom values file must be passed to both the dynamic-engine and dynamic-engine-environment helm upgrade commands. If only one chart was updated, re-run the helm upgrade for the other chart with -f custom-minimal-image-values.yaml appended.
If pods enter ImagePullBackOff, confirm that the distroless image tag exists for the deployed chart version.
The -distroless image tag is available from version 1.6.0. If you are running an earlier chart version, the distroless variant may not be present. Verify available tags:
```
crane ls ghcr.io/talend/engine-config-manager | grep distroless
```
If the tag is absent, upgrade to version 1.6.0 or later, or revert to global.profile: standard.
If you cannot execute commands inside a pod, use a debug container instead of kubectl exec.
Distroless images contain no shell. kubectl exec -it <pod> -- /bin/sh will fail. Use a debug container:
```
kubectl debug -it <pod> --image=busybox --target=<container>
```

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here