Deploying Dynamic Engine with default network policies

Enable the default,network policies for the Dynamic Engine chart and the Dynamic Engine environment chart during installation or upgrade to control traffic between the Dynamic Engine chart, the environment chart, and external services.

After setting up the default ones, you can follow the linked advanced customization scenarios to set custom egress rules.

Before you begin

The dynamic-engine-crd custom resource definitions must have been installed using the oci://ghcr.io/talend/helm/dynamic-engine-crd Helm chart. If not, run the following commands for the installation:
1. Find the chart version to be used:
  - Run the following Helm command:
    helm show chart oci://ghcr.io/talend/helm/dynamic-engine-crd --version <engine_version>
  - See the version directly from Talend Management Console or check the Dynamic Engine changelog for the chart version included in your Dynamic Engine version.
  - Use an API call to the Dynamic Engine version endpoint.
2. Run the following command to install the Helm chart of a given version:
```
helm install dynamic-engine-crd oci://ghcr.io/talend/helm/dynamic-engine-crd --version <helm_chart_version>
```
  Replace <helm_chart_version> with the chart version supported by your Dynamic Engine version.
  Without specifying the version, you install the latest available dynamic-engine-crd chart version.
The Kubernetes cluster must use a network plugin that supports network policies enforcement. For more information about Kubernetes Network Policies, see Network Policies.

About this task

This procedure applies to standard clusters and managed Kubernetes modes, provided the cluster CNI plugin enforces Kubernetes NetworkPolicy resources. You can apply it during the initial installation or during an upgrade.

The default network policy set gives you a default security baseline for Dynamic Engine deployments. For custom egress control, see Restricting egress to known IP addresses and Allowing general egress with internal restrictions.

Support for Kubernetes Network Policies was introduced in Dynamic Engine v1.6.0.

Procedure

Set the example deployment variables.

export DYNAMIC_ENGINE_VERSION=1.6.0
export DYNAMIC_ENGINE_ID=c-m-sjufu4qy
export DYNAMIC_ENGINE_ENVIRONMENT_ID=684c3baec6a6f88f61e9d59d

Replace the example values with your engine information.

Install or upgrade the Dynamic Engine and environment charts with the built-in policy set.

cat <<EOF > custom-network-policies-values.yaml
configuration:
  networkPolicies:
    enabled: true
EOF

helm upgrade --install dynamic-engine-$DYNAMIC_ENGINE_ID \
  -f $DYNAMIC_ENGINE_ID-values.yaml \
  -f custom-network-policies-values.yaml \
  oci://ghcr.io/talend/helm/dynamic-engine \
  --version $DYNAMIC_ENGINE_VERSION

helm upgrade --install dynamic-engine-environment-$DYNAMIC_ENGINE_ENVIRONMENT_ID \
  -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-values.yaml \
  -f custom-network-policies-values.yaml \
  oci://ghcr.io/talend/helm/dynamic-engine-environment \
  --version $DYNAMIC_ENGINE_VERSION

This enables the default policies for both the Dynamic Engine instance and its environment.

Verify that the policies were created.

kubectl get networkpolicies -A -l "app.qlik.com/owned-by=qlik"

The output should show the default allow and deny policies in the engine and environment namespaces.

NAMESPACE                                      NAME                                     POD-SELECTOR                                           AGE
qlik-processing-env-684c3baec6a6f88f61e9d59d   default-allow-dns                        <none>                                                 56m
qlik-processing-env-684c3baec6a6f88f61e9d59d   default-allow-egress-to-internet         <none>                                                 56m
qlik-processing-env-684c3baec6a6f88f61e9d59d   default-allow-egress-to-same-namespace   <none>                                                 56m
qlik-processing-env-684c3baec6a6f88f61e9d59d   default-deny-all                         <none>                                                 56m
qlik-dynamic-engine                            default-allow-dns                        <none>                                                 4s
qlik-dynamic-engine                            default-allow-egress-to-internet         app.kubernetes.io/name in (engine-operator,reloader)   4s
qlik-dynamic-engine                            default-allow-ingress-docker-registry    app=docker-registry                                    4s
qlik-dynamic-engine                            default-deny-all                         <none>                                                 4s

As shown in this output, network policies are scoped to the namespaces of the engine and its environments. Each namespace has its own network policies.

Results

The Dynamic Engine instance and its environment now run with the default network policies. In Talend Management Console, the environment should remain in Ready status if the policies allow the required traffic.

The default network policies for Dynamic Engine are sufficient for most pod network access control hardening needs. If you need custom egress rules, see Restricting egress to known IP addresses for Dynamic Engine or Allowing general egress with internal restrictions for Dynamic Engine.

Troubleshooting:

If pods are running but failing health checks or cannot communicate, the policies are probably too restrictive.

Verify the policies in each namespace with kubectl get networkpolicies -n <namespace> -o yaml.
Check pod logs for connection errors.
Temporarily disable the policies declaratively to verify connectivity with helm upgrade and --set configuration.networkPolicies.enabled=false. Avoid deleting Helm-managed NetworkPolicy resources directly.
Ensure the cluster uses a CNI plugin that enforces network policies.

DNS resolution failures usually mean egress to the kube-dns service is blocked.

Verify that the default-allow-dns policy is enabled.
Confirm that the policy allows egress to port 53.
If DNS still fails, add a rule that allows egress to the kube-system namespace on UDP port 53.

External service connectivity issues usually mean an egress rule is missing for a required endpoint or port.

Verify the IP addresses with dig.
Test connectivity from inside a pod with kubectl exec and curl -I https://api.us.cloud.talend.com.
Ensure the Kubernetes API server remains reachable on ports 443 or 8443.

If pods fail to pull images, the registry egress rule is probably missing or points to the wrong registry.

If you use the embedded registry, confirm that the environment policy allows egress to the docker-registry service.
If you use an external registry, add an egress rule for its IP address and port.
Verify that the registry service is running and reachable from the cluster.

If network policies are created but pods still communicate freely, the network plugin may not be enforcing policies.

Check whether the cluster uses Calico, Cilium, Weave, or another CNI that supports network policies.
Confirm that the policies are created in the correct namespaces.
Inspect the network controller pods in the kube-system namespace.

Useful debugging commands include

# View all network policies
kubectl get networkpolicies -A

# Describe a specific policy
kubectl describe networkpolicy <policy-name> -n <namespace>

# Check pod connectivity
kubectl exec <pod-name> -n <namespace> -- ping <target-pod-ip>

# Test DNS from a pod
kubectl exec <pod-name> -n <namespace> -- nslookup <service-name>

# Check pod labels (used by policies)
kubectl get pods -n <namespace> --show-labels

# Test external connectivity
kubectl exec <pod-name> -n <namespace> -- curl -I https://<external-url>

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here