Injecting environment variables from ConfigMap

Use Kubernetes ConfigMap to provide non-sensitive configuration (such as API endpoints, feature flags, or application settings) as environment variables to your Jobs (Data Integration, Big Data, and Data Services) and Routes tasks.

Before you begin

See Prerequisites and Important considerations.

Procedure

Prepare a ConfigMap resource file with environment variables.

Example

For example, create a file named configmap-external-config.yaml:

cat <<EOF > configmap-external-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: external-config
  namespace: qlik-processing-env-$DYNAMIC_ENGINE_ENVIRONMENT_ID
data:
  API_ENDPOINT: "https://api.example.com/v1"
  FEATURE_FLAG_ENABLED: "true"
  LOG_LEVEL: "INFO"
  MAX_RETRIES: "3"
EOF

Replace $DYNAMIC_ENGINE_ENVIRONMENT_ID with your environment ID and update the configuration values with your actual data.

Create a Helm values file that references the ConfigMap.

Example

Create a file named configmap-values.yaml. The content varies depending on which task types will consume this ConfigMap.

For Data Integration tasks:

configuration:
  jobDeployment:
    additionalSpec:
      enabled: true
      envFrom:
        - configMapRef:
            name: external-config

For Data Service and Route tasks:

configuration:
  dataServiceRouteDeployment:
    additionalSpec:
      enabled: true
      envFrom:
        - configMapRef:
            name: external-config

To combine both:

configuration:
  jobDeployment:
    additionalSpec:
      enabled: true
      envFrom:
        - configMapRef:
            name: external-config

  dataServiceRouteDeployment:
    additionalSpec:
      enabled: true
      envFrom:
        - configMapRef:
            name: external-config

Tip: The additionalSpec.envFrom.configMapRef structure loads all keys from the ConfigMap as environment variables. All keys become available with their exact names specified in the ConfigMap, such as API_ENDPOINT or FEATURE_FLAG_ENABLED.

Deploy or upgrade your Dynamic Engine environment to create the required namespace, if this has not been done previously.
```
helm upgrade --install dynamic-engine-environment-$DYNAMIC_ENGINE_ENVIRONMENT_ID \
  oci://ghcr.io/talend/helm/dynamic-engine-environment \
  --version ${DYNAMIC_ENGINE_VERSION} \
  -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-values.yaml
```
Replace ${DYNAMIC_ENGINE_VERSION} with your environment's version.

This command creates the Dynamic Engine environment and its associated namespace qlik-processing-env-$DYNAMIC_ENGINE_ENVIRONMENT_ID.
Create the ConfigMap in your Dynamic Engine environment namespace.
```
kubectl apply -f configmap-external-config.yaml
```
This command creates the ConfigMap resource in your Kubernetes cluster. The ConfigMap is available for use by all pods in the environment namespace.
Upgrade your Dynamic Engine environment with the ConfigMap configuration.
```
helm upgrade dynamic-engine-environment-$DYNAMIC_ENGINE_ENVIRONMENT_ID \
  oci://ghcr.io/talend/helm/dynamic-engine-environment \
  --version ${DYNAMIC_ENGINE_VERSION} \
  -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-values.yaml \
  -f configmap-values.yaml
```
Replace ${DYNAMIC_ENGINE_VERSION} with your environment's version.

This command applies your ConfigMap configuration to the Dynamic Engine environment. The environment variables become available to all deployed tasks.

Verify that environment variables from the ConfigMap are available in running pods.

kubectl get pod -n qlik-processing-env-$DYNAMIC_ENGINE_ENVIRONMENT_ID <pod-name> \
  -o jsonpath='{.spec.containers[*].envFrom}'

Expected output should include the ConfigMap reference:

[{"configMapRef":{"name":"external-config"}}]

Results

Your Dynamic Engine environment provides access to configuration from your Kubernetes ConfigMap as environment variables.

When you update the variables in your ConfigMap, the changes are applied as follows:

Jobs (Data Integration, including Big Data tasks) are one-time tasks. Updated values are read when the next task run starts.
Routes and Data Services are always-on tasks; they do not automatically reload ConfigMap changes. Update the task in Talend Management Console to trigger a re-deployment in your Dynamic Engine environment.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here