Injecting environment variables from Kubernetes Secret

Inject sensitive data (passwords, API keys, tokens) from Kubernetes Secret as environment variables in your Job (Data Integration, Big Data, and Data Services) and Routes tasks so they can easily and securely access credentials.

Important: Your sensitive data is stored in your Kubernetes Secret and managed by your organization's security policy. The Dynamic Engine customization pattern only provides the access to this information to the tasks deployed in your Dynamic Engine environments. Qlik does not manage your sensitive data.

Before you begin

See Prerequisites and Important considerations.

Procedure

Create a Kubernetes Secret resource file with sensitive data.
Create a file named external-secret-as-env.yaml with your sensitive data:
```
cat <<EOF > external-secret-as-env.yaml
apiVersion: v1
kind: Secret
metadata:
  name: external-secret-as-env
  namespace: qlik-processing-env-$DYNAMIC_ENGINE_ENVIRONMENT_ID
type: Opaque
stringData:
  DATABASE_PASSWORD: "mySecurePassword123"
  API_KEY: "your_key"
  AUTH_TOKEN: "Bearer your_token"
EOF
```
Replace the example credentials with your actual sensitive data.

Dynamic Engine does not manage your sensitive data. For further information on how the data is encoded in a Kubernetes Secret, see Secret from the Kubernetes documentation.
Create a Helm values file that references the Secret.

Create a file named, for example, secret-values.yaml. Choose the approach that matches your use case:
- Inject all Secret keys as environment variables using additionalSpec.envFrom (bulk injection):
```
configuration:
  jobDeployment:
    additionalSpec:
      enabled: true
      envFrom:
        - secretRef:
            name: external-secret-as-env
```
  This approach loads all keys from the Secret as environment variables: DATABASE_PASSWORD, API_KEY, and AUTH_TOKEN.
- Selectively inject specific Secret keys using additionalSpec.env (fine-grained control):
```
configuration:
  jobDeployment:
    additionalSpec:
      enabled: true
      env:
        - name: DATABASE_PASSWORD
          valueFrom:
            secretKeyRef:
              name: external-secret-as-env
              key: DATABASE_PASSWORD
        - name: API_KEY
          valueFrom:
            secretKeyRef:
              name: external-secret-as-env
              key: API_KEY
```
  This approach selectively injects only the keys you need, leaving other Secret keys inaccessible to the pod.
Deploy or upgrade your Dynamic Engine environment to create the required namespace.
```
helm upgrade --install dynamic-engine-environment-$DYNAMIC_ENGINE_ENVIRONMENT_ID \
  oci://ghcr.io/talend/helm/dynamic-engine-environment \
  --version ${DYNAMIC_ENGINE_VERSION} \
  -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-values.yaml
```
Replace ${DYNAMIC_ENGINE_VERSION} with your environment's version.

This command creates the Dynamic Engine environment and its associated namespace qlik-processing-env-$DYNAMIC_ENGINE_ENVIRONMENT_ID.
Create the Secret in your Dynamic Engine environment namespace.
```
kubectl apply -f external-secret-as-env.yaml
```
This command creates the Secret resource in your Kubernetes cluster. The Secret is now available for use by all pods in the environment namespace.
Upgrade your Dynamic Engine environment with the Secret configuration.
```
helm upgrade dynamic-engine-environment-$DYNAMIC_ENGINE_ENVIRONMENT_ID \
  oci://ghcr.io/talend/helm/dynamic-engine-environment \
  --version ${DYNAMIC_ENGINE_VERSION} \
  -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-values.yaml \
  -f secret-values.yaml
```
Replace ${DYNAMIC_ENGINE_VERSION} with your environment's version.

This command applies your Secret configuration to the Dynamic Engine environment. Sensitive data is now available as environment variables in all deployed tasks.
Verify that Secret environment variables are available in running pods.
Verify that the Secret is correctly injected using the command that matches your chosen injection method:

If you used bulk injection with envFrom:
```
kubectl get pod -n qlik-processing-env-$DYNAMIC_ENGINE_ENVIRONMENT_ID <pod-name> \
  -o jsonpath='{.spec.containers[*].envFrom}'
```
Expected output confirms the Secret reference:
```
[{"secretRef":{"name":"external-secret-as-env"}}]
```
If you used selective injection with env:
```
kubectl get pod -n qlik-processing-env-$DYNAMIC_ENGINE_ENVIRONMENT_ID <pod-name> \
  -o jsonpath='{.spec.containers[*].env}'
```
Expected output shows your selected Secret keys are mounted as environment variables.

Results

Tasks in your Dynamic Engine environment now have secure access to sensitive credentials from Kubernetes Secret. You can rotate credentials by updating the Secret, and the changes are applied as follows:

Data Integration (including Big Data) tasks are one-time tasks. Updated values are read when the next task run starts.
Routes and Data Services are always-on tasks; they do not automatically reload Secret changes. Update the task in Talend Management Console to trigger a re-deployment in your Dynamic Engine environment.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here