Configuring custom truststore or keystore files for Jobs and Routes

Configure custom truststore or keystore files for data integration tasks (including big data), Data Services, and Routes to authenticate their connections to external systems.

In this example, data integration Jobs are used to demonstrate the configuration. The same procedure applies to Data Services and Routes, too.

Before you begin

The dynamic-engine-crd custom resource definitions must have been installed using the oci://ghcr.io/talend/helm/dynamic-engine-crd Helm chart. If not, run the following commands for the installation:
1. Find the chart version to be used:
  - Run the following Helm command:
    helm show chart oci://ghcr.io/talend/helm/dynamic-engine-crd --version <engine_version>
  - See the version directly from Talend Management Console or check the Dynamic Engine changelog for the chart version included in your Dynamic Engine version.
  - Use an API call to the Dynamic Engine version endpoint.
2. Run the following command to install the Helm chart of a given version:
```
helm install dynamic-engine-crd oci://ghcr.io/talend/helm/dynamic-engine-crd --version <helm_chart_version>
```
  Replace <helm_chart_version> with the chart version supported by your Dynamic Engine version.
  Without specifying the version, you install the latest available dynamic-engine-crd chart version.
Your Dynamic Engine must have been deployed. If not, run this command to deploy it:
```
helm install dynamic-engine -f <engine-id>-helm-values/<engine-id>-values.yaml oci://ghcr.io/talend/helm/dynamic-engine 
```
This is the default deployment, because the current customization is only needed on the Dynamic Engine environment side. For this reason no customization is configured for the Dynamic Engine instance.

About this task

When your Jobs or Routes need to connect to external systems that require TLS-based authentication, you can provide a custom truststore to validate the external system's certificate, or both a truststore and keystore if the external system requires client certificate authentication.

Note: Routes or Data Services:

This configuration applies only to outbound connections to external systems. It does not enable HTTPS for incoming client requests to Routes or Data Services. The keystore and truststore configured here are used when the components in your Routes or Data Services act as TLS clients to external systems, not for exposing the components themselves over HTTPS.

Procedure

In the Kubernetes machine, unzip the Helm deployment zip file previously downloaded.
Create the truststore or keystore file to be used by your data integration Jobs.
Example
For example, your Jobs need to connect to a MySQL database and need their custom truststore to verify the server's identity. You can use the keytool command-line utility to generate truststore and keystore files. Then the truststore used by your Jobs must contain the Certificate Authority (CA) certificate that signed the MySQL server’s certificate, so your Dynamic Engine environment as a MySQL client can validate the server’s identity.

For instructions on keytool, see Keytool. For instructions on setting up MySQL's SSL connection, see MySQL's documentation.
Information noteNote: About keystore and truststore:
- Server keystore and client truststore: The server keystore holds the server’s private key and certificate chain. The client truststore must contain either the server’s signed certificate (exported from the server keystore using keytool -export) or the Certificate Authority (CA) certificate that signed the server’s certificate, so the client can validate the server’s identity.
  At runtime, the client’s TLS (Transport Layer Security) stack verifies whether it can trust a server's certificate. To do this, it follows the server's certificate chain, through any intermediate certificates, until it finds a certificate already trusted in its truststore. If this verification fails, the secure connection (handshake) is aborted. In other words, the client only connects when it matches the server's certificate with an entry in its truststore.

Create a custom values file to provide the truststore or both the truststore and keystore files to the Dynamic Engine environment.

Example

In this example, only the truststore file is used.

STORE_TYPE=truststore
STORE_FILE=${keystoreFolder}/job-truststore.pc12
STORE_PASSWORD=${jobTruststorePassword}
STORE_FILE_TYPE=$(keytool -list -keystore "$STORE_FILE" \
-storepass "$STORE_PASSWORD" 2>/dev/null | awk -F': ' '/Keystore type/{print $2}')
cat <<EOF > $DYNAMIC_ENGINE_ENVIRONMENT_ID-custom-$STORE_TYPE-values.yaml
configuration:
  $STORE_TYPE:
    enabled: true
    data: $(base64 -i $STORE_FILE | tr -d '\n')
    password: $STORE_PASSWORD
    type: $STORE_FILE_TYPE
EOF

Replace the variable values as needed for your environment and credentials.

Tip: In some use cases, your Jobs may need to provide both a truststore (to verify the identity of the external system) and a keystore (to authenticate themselves to that external system). For those scenarios, create a values file for each of the truststore and keystore:

# Create the truststore values file
STORE_TYPE=truststore
STORE_FILE=${keystoreFolder}/job-truststore.pc12
STORE_PASSWORD=${jobTruststorePassword}
STORE_FILE_TYPE=$(keytool -list -keystore "$STORE_FILE" \
-storepass "$STORE_PASSWORD" 2>/dev/null | awk -F': ' '/Keystore type/{print $2}')
cat <<EOF > $DYNAMIC_ENGINE_ENVIRONMENT_ID-custom-$STORE_TYPE-values.yaml
configuration:
  $STORE_TYPE:
    enabled: true
    data: $(base64 -i $STORE_FILE | tr -d '\n')
    password: $STORE_PASSWORD
    type: $STORE_FILE_TYPE
EOF  

# Create the values file forkeystore
STORE_TYPE=keystore
STORE_FILE=${keystoreFolder}/job-keystore.jks
STORE_PASSWORD=${jobKeystorePassword}
STORE_FILE_TYPE=$(keytool -list -keystore "$STORE_FILE" \
-storepass "$STORE_PASSWORD" 2>/dev/null | awk -F': ' '/Keystore type/{print $2}')
cat <<EOF > $DYNAMIC_ENGINE_ENVIRONMENT_ID-custom-$STORE_TYPE-values.yaml
configuration:
  $STORE_TYPE:
    enabled: true
    data: $(base64 -i $STORE_FILE | tr -d '\n')
    password: $STORE_PASSWORD
    type: $STORE_FILE_TYPE
EOF

Install or upgrade the Dynamic Engine environment chart with the custom values file.

helm install dynamic-engine-environment-$DYNAMIC_ENGINE_ENVIRONMENT_ID -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-values.yaml  \
 -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-custom-truststore-values.yaml \
 oci://ghcr.io/talend/helm/dynamic-engine-environment \
 --version $DYNAMIC_ENGINE_VERSION

As explained earlier, only the truststore values file is needed here.

For upgrades, use helm upgrade instead of helm install.

Tip: When the Jobs need to provide both the truststore and keystore files, run this command to apply both values files at the same time:

helm install dynamic-engine-environment-$DYNAMIC_ENGINE_ENVIRONMENT_ID -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-values.yaml  \
 -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-custom-truststore-values.yaml \
  -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-custom-keystore-values.yaml \
 oci://ghcr.io/talend/helm/dynamic-engine-environment \
 --version $DYNAMIC_ENGINE_VERSION

Verify that the Dynamic Engine environment is ready and the data integration Jobs can access the custom truststore or keystore files.
- Verify that all environment services are running.
- Launch a data integration Job and confirm that it connects to the required external systems using the custom truststore or keystore files.

Results

When the deployment is completed, your data integration Jobs in the Dynamic Engine environment use the specified truststore or keystore for secure connectivity.

Tip: To connect data integration Jobs securely to external systems, you only need to configure the truststore or keystore file at the environment level. This file contains the certificate chain that allows TLS authentication to external systems. You do not need to change anything in the Jobs or in Studio.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here