Security with a custom keystore for Dynamic Engine environment services

Use a custom keystore for your Dynamic Engine environment to verify signed data integration Job artifacts.

For details about generating and managing keystores in Talend Studio, see Configuring custom Java KeyStore for Job artifact signature.

Important: Setting a custom keystore for Dynamic Engine environment services is only allowed during engine deployment or upgrade.

Before you begin

The dynamic-engine-crd custom resource definitions must have been installed using the oci://ghcr.io/talend/helm/dynamic-engine-crd helm chart. If not, run the following commands for the installation:
1. Find the chart version to be used:
  - Run the following Helm command:
    helm show chart oci://ghcr.io/talend/helm/dynamic-engine-crd --version <engine_version>
  - See the version directly from Talend Management Console or check the Dynamic Engine changelog for the chart version included in your Dynamic Engine version.
  - Use an API call to the Dynamic Engine version endpoint.
2. Run the following command to install the Helm chart of a given version:
```
helm install dynamic-engine-crd oci://ghcr.io/talend/helm/dynamic-engine-crd --version <helm_chart_version>
```
  Replace <helm_chart_version> with the chart version supported by your Dynamic Engine version.
  Without specifying the version, you install the latest available dynamic-engine-crd chart version.
Your Dynamic Engine must have been deployed. If not, run this command to deploy it:
```
helm install dynamic-engine -f <engine-id>-helm-values/<engine-id>-values.yaml oci://ghcr.io/talend/helm/dynamic-engine 
```
This is the default deployment, because the current customization is only needed on the Dynamic Engine environment side. For this reason no customization is configured for the Dynamic Engine instance.

About this task

By default, a Dynamic Engine environment uses the default keystore for artifact signature verification. To verify data integration Job artifacts signed with your own keystore, you must provide the custom keystore and its password when you deploy or upgrade the environment.

You only need to include the public certificate of the root certificate authority (CA) in your custom keystore. The private keys or intermediate certificates are not necessary.

Procedure

In the Kubernetes machine, unzip the Helm deployment zip file previously downloaded.
Set the KEYSTORE_PATH and KEYSTORE_PASS environment variables for the location of your keystore file and its access password.
Example
For example, for a Linux system:
```
export KEYSTORE_PATH=<location-to-your-keystore-file>
export KEYSTORE_PASS=<your_keystore_password>
```
Replace <location-to-your-keystore-file> and <your_keystore_password> with actual values.

Create a custom values file for your Dynamic Engine environment with the base64-encoded keystore and its password.

cat <<EOF > $DYNAMIC_ENGINE_ENVIRONMENT_ID-custom-keystore-values.yaml
configuration:
  signature:
    keystore:
      # base64 string of the keystore to load, null by default (rely on the default signature bundle)
      data: |
        $(base64 -i $KEYSTORE_PATH)
      password: ${KEYSTORE_PASS}
EOF

Replace $DYNAMIC_ENGINE_ENVIRONMENT_ID with your environment ID.

Deploy the Dynamic Engine environment with your custom keystore configuration.

helm install dynamic-engine-environment-$DYNAMIC_ENGINE_ENVIRONMENT_ID -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-values.yaml  \
 -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-custom-keystore-values.yaml \
 oci://ghcr.io/talend/helm/dynamic-engine-environment \
 --version $DYNAMIC_ENGINE_VERSION

Tip: When changes are needed, you can update the content of the $DYNAMIC_ENGINE_ENVIRONMENT_ID-custom-keystore-values.yaml file and run helm upgrade to apply the changes:

helm upgrade dynamic-engine-environment-$DYNAMIC_ENGINE_ENVIRONMENT_ID -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-values.yaml  \
 -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-custom-keystore-values.yaml \
 oci://ghcr.io/talend/helm/dynamic-engine-environment \
 --version $DYNAMIC_ENGINE_VERSION

Results

Once done successfully, the Dynamic Engine environment services are installed with the custom keystore in your cluster and connect to the Dynamic Engine.

In Talend Management Console, the status of this environment becomes Ready, confirming that it is ready to run tasks or plans.

When a data integration task runs, the artifact signature is verified against the custom keystore first, then the default one. If neither can authenticate the signature, execution does not start and the task appears as failed in Talend Management Console.

If task execution fails due to signature issues, create a simple Pod template to verify the custom keystore in the environment:

Create the values file for this test pod. In this example, the file is called check-custom-keystore.yaml.

cat <<EOF > check-custom-keystore.yaml
apiVersion: v1
kind: Pod
metadata:
  labels:
    app.kubernetes.io/managed-by: kubectl
  name: check-custom-keystore
  
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 61000
    seccompProfile:
      type: RuntimeDefault
  containers:
  - name: runner
    image: ghcr.io/talend/jre21-base-minimal:4.0.32
    command:
      - cat
    tty: true
    securityContext:
      allowPrivilegeEscalation: false
      capabilities:
        drop:
          - ALL
    resources:
      requests:
        cpu: '0.3'
        memory: 256Mi
      limits:
        cpu: '0.5'
        memory: 256Mi
    volumeMounts:
    - mountPath: /tmp/signature/keystores/default
      name: signature-check-secret-volume
      readOnly: true
    - mountPath: /tmp/signature/keystores/custom
      name: custom-signature-check-secret-volume
      readOnly: true
  volumes:
    - name: signature-check-secret-volume
      secret:
        defaultMode: 420
        secretName: signature-check-secret
    - name: custom-signature-check-secret-volume
      secret:
        defaultMode: 420
        secretName: custom-signature-check-secret
EOF

Apply the test Pod template:

kubectl apply -n qlik-processing-env-$DYNAMIC_ENGINE_ENVIRONMENT_ID -f check-custom-keystore.yaml

List keystore contents with keytool:

kubectl exec -it check-custom-keystore -n qlik-processing-env-$DYNAMIC_ENGINE_ENVIRONMENT_ID \
-- /usr/lib/jvm/jdk-21.0.5+11-jre/bin/keytool -list -v -keystore /tmp/signature/keystores/custom/custom-code-verify.jks \
-storepass \$(cat /tmp/signature/keystores/custom/custom-code-verify-jks-password)

This command confirms the custom keystore is mounted and readable in the environment.

Delete the test Pod when finished:

 kubectl delete -n qlik-processing-env-$DYNAMIC_ENGINE_ENVIRONMENT_ID -f check-custom-keystore.yaml

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here