Skip to main content Skip to complementary content
Qlik Resources
Loading SearchUnify's search If you need assistance with your product, please contact Qlik Support.
Qlik Customer Portal
Loading SearchUnify's search If you need assistance with your product, please contact Qlik Support.
Qlik Customer Portal

Development environment only: Configuring internal Docker registry for autoscaling

Configure the internal Docker registry for autoscaling in quick-start or legacy deployments only. In a production environment, use an external Docker registry instead.

About this task

Dynamic Engine ships with an internal Docker registry intended for quick-start deployments. This registry is not recommended for production use and should not be customized for autoscaling in production environments. For production deployments, configure an external Docker registry instead. See Configuring a custom Docker registry for Talend Management Console tasks for recommended configuration.

If you are using legacy deployments with autoscaling enabled on the internal registry, this procedure describes how to configure it. In the example below, you activate HPA on the internal Docker registry and set a custom value for the number of replicas to be created and the maximum number of replicas. HPA stands for horizontal pod autoscaling, a Kubernetes feature that automatically adjusts the number of pods in response to demand. A pod is a Kubernetes object that represents a group of one or more containers.

Information noteImportant:

If you use autoscaling with the internal Docker registry, you must keep persistent shared storage enabled. Set configuration.persistence.enabled=true in your deployment values file. Without persistent shared storage, autoscaling on the internal registry will fail. This is a critical requirement for the internal registry autoscaling to function.

Procedure

  1. Create a custom values file for autoscaling and replica settings: 
    cat <<EOF > $DYNAMIC_ENGINE_ID-custom-internal-registry.yaml
configuration:
  dockerRegistry:
    replicaCount: 2
    autoscaling:
      enabled: true
      maxReplicas: 5
EOF
  2. Apply this custom values file: 
    helm install -i dynamic-engine-$DYNAMIC_ENGINE_ID -f $DYNAMIC_ENGINE_ID-values.yaml \
            -f $DYNAMIC_ENGINE_ID-custom-internal-registry.yaml \
            oci://ghcr.io/talend/helm/dynamic-engine \
            --version $DYNAMIC_ENGINE_VERSION

    If you are upgrading the engine, use the following command:

    helm upgrade -i dynamic-engine-$DYNAMIC_ENGINE_ID -f $DYNAMIC_ENGINE_ID-values.yaml \
            -f $DYNAMIC_ENGINE_ID-custom-internal-registry.yaml \
            oci://ghcr.io/talend/helm/dynamic-engine \
            --version $DYNAMIC_ENGINE_VERSION

What to do next

Before you deploy or upgrade your Dynamic Engine instance, verify that persistent shared storage is enabled in your values file: configuration.persistence.enabled=true. This is required for autoscaling to function properly.

Apply the values files (both the main values file and this custom autoscaling file) during installation or upgrade:

helm install -i dynamic-engine-$DYNAMIC_ENGINE_ID -f $DYNAMIC_ENGINE_ID-values.yaml \
  -f $DYNAMIC_ENGINE_ID-custom-internal-registry.yaml \
  oci://ghcr.io/talend/helm/dynamic-engine \
  --version $DYNAMIC_ENGINE_VERSION

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here