Configuring custom truststore or keystore files for data integration Jobs

Provide custom truststore or keystore file for data integration Jobs running in your Dynamic Engine environment to enable secure connectivity.

About this task

To connect securely to external services for your Job, you can provide your own truststore, or both a truststore and a keystore, during environment deployment.

Procedure

Create the truststore or keystore file to be used by your data integration Jobs.
Example
For example, your Jobs need to connect to a MySQL database and need their custom truststore to verify the server's identity. You can use the keytool command-line utility to generate truststore and keystore files. Then the truststore used by your Jobs must contain the Certificate Authority (CA) certificate that signed the MySQL server’s certificate, so your Dynamic Engine environment as a MySQL client can validate the server’s identity.

For instructions on keytool, see Keytool. For instructions on setting up MySQL's SSL connection, see MySQL's documentation.
Information noteNote: About keystore and truststore:
- Server keystore and client truststore: The server keystore holds the server’s private key and certificate chain. The client truststore must contain either the server’s signed certificate (exported from the server keystore using keytool -export) or the Certificate Authority (CA) certificate that signed the server’s certificate, so the client can validate the server’s identity.
  At runtime, the client’s TLS (Transport Layer Security) stack verifies whether it can trust a server's certificate. To do this, it follows the server's certificate chain, through any intermediate certificates, until it finds a certificate already trusted in its truststore. If this verification fails, the secure connection (handshake) is aborted. In other words, the client only connects when it matches the server's certificate with an entry in its truststore.

Create a custom values file to provide the truststore or both the truststore and keystore files to the Dynamic Engine environment.

Example

In this example, only the truststore file is used.

DYN_ENGINE_ENV_ID=<environment-id>
STORE_TYPE=truststore
STORE_FILE=${keystoreFolder}/job-truststore.pc12
STORE_PASSWORD=${jobTruststorePassword}
STORE_FILE_TYPE=$(keytool -list -keystore "$STORE_FILE" \
-storepass "$STORE_PASSWORD" 2>/dev/null | awk -F': ' '/Keystore type/{print $2}')
cat <<EOF > $DYN_ENGINE_ENV_ID-custom-$STORE_TYPE-values.yaml
configuration:
  $STORE_TYPE:
    enabled: true
    data: $(base64 -i $STORE_FILE | tr -d '\n')
    password: $STORE_PASSWORD
    type: $STORE_FILE_TYPE
EOF

Replace the variable values as needed for your environment and credentials.

Tip: In some use cases, your Jobs may need to provide both a truststore (to verify the identity of the external system) and a keystore (to authenticate themselves to that external system). For those scenarios, create a values file for each of the truststore and keystore:

# Create the truststore values file
DYN_ENGINE_ENV_ID=<environment-id>
STORE_TYPE=truststore
STORE_FILE=${keystoreFolder}/job-truststore.pc12
STORE_PASSWORD=${jobTruststorePassword}
STORE_FILE_TYPE=$(keytool -list -keystore "$STORE_FILE" \
-storepass "$STORE_PASSWORD" 2>/dev/null | awk -F': ' '/Keystore type/{print $2}')
cat <<EOF > $DYN_ENGINE_ENV_ID-custom-$STORE_TYPE-values.yaml
configuration:
  $STORE_TYPE:
    enabled: true
    data: $(base64 -i $STORE_FILE | tr -d '\n')
    password: $STORE_PASSWORD
    type: $STORE_FILE_TYPE
EOF  

# Create the values file forkeystore
STORE_TYPE=keystore
STORE_FILE=${keystoreFolder}/job-keystore.jks
STORE_PASSWORD=${jobKeystorePassword}
STORE_FILE_TYPE=$(keytool -list -keystore "$STORE_FILE" \
-storepass "$STORE_PASSWORD" 2>/dev/null | awk -F': ' '/Keystore type/{print $2}')
cat <<EOF > $DYN_ENGINE_ENV_ID-custom-$STORE_TYPE-values.yaml
configuration:
  $STORE_TYPE:
    enabled: true
    data: $(base64 -i $STORE_FILE | tr -d '\n')
    password: $STORE_PASSWORD
    type: $STORE_FILE_TYPE
EOF

Install or upgrade the Dynamic Engine environment chart with the custom values file.

helm install dyn-engine-env-<environment-id> -f <environment-id>-values.yaml  \
 -f <environment-id>-custom-truststore-values.yaml \
 oci://ghcr.io/talend/helm/dynamic-engine-environment \
 --version <engine-version>

As explained earlier, only the truststore values file is needed here.

For upgrades, use helm upgrade instead of helm install.

Tip: When the Jobs need to provide both the truststore and keystore files, run this command to apply both values files at the same time:

helm install dyn-engine-env-<environment-id> -f <environment-id>-values.yaml  \
 -f <environment-id>-custom-truststore-values.yaml \
  -f <environment-id>-custom-keystore-values.yaml \
 oci://ghcr.io/talend/helm/dynamic-engine-environment \
 --version <engine-version>

Verify that the Dynamic Engine environment is ready and the data integration Jobs can access the custom truststore or keystore files.
- Verify that all environment services are running.
- Launch a data integration Job and confirm that it connects to the required external systems using the custom truststore or keystore files.

Results

When the deployment is completed, your data integration Jobs in the Dynamic Engine environment use the specified truststore or keystore for secure connectivity.

Tip: To connect data integration Jobs securely to external systems, you only need to configure the truststore or keystore file at the environment level. This file contain the certificate chain that allows SSL connections. You do not need to change anything in the Jobs or in Studio.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here