Using an HTTP proxy for Dynamic Engine services in air-gapped environments

Configure Dynamic Engine to use an HTTP proxy for all HTTP/HTTPS requests from Dynamic Engine services. This is required for air-gapped environments where outbound traffic must go through a proxy.

Before you begin

Obtain the HTTP/HTTPS proxy server address and port.
The dynamic-engine-crd custom resource definitions must have been installed using the oci://ghcr.io/talend/helm/dynamic-engine-crd helm chart. If not, run the following commands for the installation:
1. Find the chart version to be used:
  - Run the following Helm command:
    helm show chart oci://ghcr.io/talend/helm/dynamic-engine-crd --version <engine_version>
  - See the version directly from Talend Management Console or check the Dynamic Engine changelog for the chart version included in your Dynamic Engine version.
  - Use an API call to the Dynamic Engine version endpoint.
2. Run the following command to install the Helm chart of a given version:
```
helm install dynamic-engine-crd oci://ghcr.io/talend/helm/dynamic-engine-crd --version <helm_chart_version>
```
  Replace <helm_chart_version> with the chart version supported by your Dynamic Engine version.
  Without specifying the version, you install the latest available dynamic-engine-crd chart version.

About this task

This procedure describes how to configure Dynamic Engine to use an HTTP proxy for all HTTP/HTTPS requests from Dynamic Engine services. This configuration is required for deployments in air-gapped environments, where direct internet access is restricted and outbound traffic must go through a proxy.

Procedure

Create custom values files for Dynamic Engine and its environment for the HTTP proxy.

cat <<EOF > $DYNAMIC_ENGINE_ID-custom-http-proxy-values.yaml
global:
  proxy:
    enabled: true
    httpProxy: http://<proxy-server-address>[:<port>]
    httpsProxy: http[s]://<proxy-server-address>[:<port>]
EOF

cat <<EOF > $DYNAMIC_ENGINE_ENVIRONMENT_ID-custom-http-proxy-values.yaml
global:
  proxy:
    enabled: true
    httpProxy: http://<proxy-server-address>[:<port>]
    httpsProxy: http[s]://<proxy-server-address>[:<port>]
EOF

To activate the HTTP proxy, set enabled: true.
To deactivate the HTTP proxy, set enabled: false.
To block all HTTP traffic for external calls, set httpProxy: ~ (null deactivates proxy usage).
The port is optional.
The scheme can be http or https.

Install or upgrade Dynamic Engine and its environment with HTTP proxy.

helm upgrade dynamic-engine-$DYNAMIC_ENGINE_ID --install \
  -f $DYNAMIC_ENGINE_ID-values.yaml  \
  -f $DYNAMIC_ENGINE_ID-custom-http-proxy-values.yaml \
  oci://ghcr.io/talend/helm/dynamic-engine \
  --version $DYNAMIC_ENGINE_VERSION

helm upgrade dynamic-engine-environment-$DYNAMIC_ENGINE_ENVIRONMENT_ID --install \
  -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-values.yaml  \
  -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-custom-http-proxy-values.yaml \
  oci://ghcr.io/talend/helm/dynamic-engine-environment \
  --version $DYNAMIC_ENGINE_VERSION

Results

In the dynamic-engine namespace, the dynamic-engine-operator service is able to call cloud endpoints.

In the dynamic-engine-environment namespace, services that communicate with cloud endpoints are not blocked.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here