# Configuring PodDisruptionBudget for Dynamic Engine deployments

Enable, disable, and tune PodDisruptionBudget settings in the instance and environment Helm charts.

In Kubernetes, a PodDisruptionBudget object defines how many pods can be voluntarily disrupted at the same time. For background information, see [Pod disruptions](https://kubernetes.io/docs/concepts/workloads/pods/disruptions).

PodDisruptionBudget helps keep services available during voluntary disruptions such as node maintenance or cluster scaling. Apply this procedure during an initial deployment or when you upgrade an existing release.

Support for Kubernetes PodDisruptionBudget was introduced in [Dynamic Engine v1.6.0](https://help.qlik.com/talend/en-US/dynamic-engine-changelog/Cloud/dynamic-engine-v1-6-0-release-changelog).

## Before you begin

* Your Kubernetes cluster supports PodDisruptionBudget resources.
* You have basic knowledge of [Kubernetes PodDisruptionBudget](https://kubernetes.io/docs/tasks/run-application/configure-pdb/).

## About this task

Information noteImportant:

If you plan to use EKS Auto Mode or GKE Autopilot, it is recommended to first set up PodDisruptionBudget before you deploy. Use it with global.profile: minimal and configuration.persistence.enabled=false.

Data Integration Jobs use the global PodDisruptionBudget setting by default and do not expose a separate task-level setting. Task-level customization is available for Data Service and Route tasks only.

A Pod Disruption Budget (PDB) is usually useful for:

* Protecting critical services from being evicted during cluster maintenance
* Ensuring minimum pod availability quotas (minimum available or maximum unavailable pods)
* Balancing between availability and cluster operational flexibility
* Avoiding cascading failures during coordinated pod disruptions

## Procedure

1. Set the example deployment variables.
   ```
   DYNAMIC_ENGINE_VERSION=1.6.0
   DYNAMIC_ENGINE_ID=c-m-sjufu4qy
   DYNAMIC_ENGINE_ENVIRONMENT_ID=684c3baec6a6f88f61e9d59d
   ```

   Replace the example values with the identifiers for your deployment.
2. Create a custom values file that enables PodDisruptionBudget globally.

   By default, the charts do not create PodDisruptionBudget resources. Set global.pdb.enabled to true to create the default PDB resources for both charts.

   Set global.pdb.enabled to true before you customize configuration.dataServiceRouteDeployment.podDisruptionBudget. The environment chart reads those values only when the global flag is enabled.
   ```
   cat <<EOF > custom-pdb-values.yaml
   global:
     pdb:
       enabled: true
   EOF
   ```

   This file enables the default PDB resources for the Dynamic Engine instance and its environments. This global configuration has a maximum unavailable pods constraint set to 0. This is also the default PDB rule for Data Service and Route tasks in the environment chart. Data Integration Jobs use this global PDB rule only.
3. Install or upgrade the Dynamic Engine instance and environment charts with the global PDB values file.
   ```
   helm upgrade --install dynamic-engine-$DYNAMIC_ENGINE_ID \
    -f $DYNAMIC_ENGINE_ID-values.yaml  \
    -f custom-pdb-values.yaml \
    oci://ghcr.io/talend/helm/dynamic-engine \
    --version $DYNAMIC_ENGINE_VERSION

   helm upgrade --install dynamic-engine-environment-$DYNAMIC_ENGINE_ENVIRONMENT_ID \
    -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-values.yaml  \
    -f custom-pdb-values.yaml \
    oci://ghcr.io/talend/helm/dynamic-engine-environment \
    --version $DYNAMIC_ENGINE_VERSION
   ```
4. Set custom PodDisruptionBudget rules for Data Service and Route tasks, if needed.
   1. Create a custom values file with your Data Service and Route PDB rules.
      ```
      cat <<EOF > custom-pdb-data-services-routes-values.yaml
      global:
        pdb:
          enabled: true
      configuration:
        dataServiceRouteDeployment:
          podDisruptionBudget:
            maxUnavailable: 1
            minAvailable: ~
            unhealthyPodEvictionPolicy: AlwaysAllow # or IfHealthyBudget
      EOF
      ```
      * Properties maxUnavailable and minAvailable are mutually exclusive. Set minAvailable to ~ when you set maxUnavailable.

        If you prefer to keep a minimum number of pods available, set minAvailable instead and leave maxUnavailable as ~.

      * Use unhealthyPodEvictionPolicy to control when Kubernetes can evict unhealthy pods. The accepted values are AlwaysAllow or IfHealthyBudget.

        For details about the unhealthy pod eviction policy, see the [Unhealthy Pod Eviction Policy](https://kubernetes.io/docs/tasks/run-application/configure-pdb/#unhealthy-pod-eviction-policy) section in the Kubernetes documentation.
   2. Upgrade the environment chart with the custom Data Service and Route PDB rules.
      ```
      helm upgrade --install dynamic-engine-environment-$DYNAMIC_ENGINE_ENVIRONMENT_ID \
       -f $DYNAMIC_ENGINE_ENVIRONMENT_ID-values.yaml  \
       -f custom-pdb-values.yaml \
       -f custom-pdb-data-services-routes-values.yaml \
       oci://ghcr.io/talend/helm/dynamic-engine-environment \
       --version $DYNAMIC_ENGINE_VERSION
      ```
5. Temporarily disable PodDisruptionBudget when you need unrestricted maintenance.

   Run the following commands to disable PDB without changing the rest of the release values.
   ```
   helm upgrade dynamic-engine-$DYNAMIC_ENGINE_ID \
   oci://ghcr.io/talend/helm/dynamic-engine --version $DYNAMIC_ENGINE_VERSION \
   --reuse-values --set global.pdb.enabled=false

   helm upgrade dynamic-engine-environment-$DYNAMIC_ENGINE_ENVIRONMENT_ID \
   oci://ghcr.io/talend/helm/dynamic-engine-environment --version $DYNAMIC_ENGINE_VERSION \
   --reuse-values --set global.pdb.enabled=false
   ```

## Results

The Dynamic Engine instance and its environment now use the selected PodDisruptionBudget settings. Use the following commands to confirm the active Helm values:

```
helm get values dynamic-engine-$DYNAMIC_ENGINE_ID --output json | jq -r '.global.pdb'
helm get values dynamic-engine-environment-$DYNAMIC_ENGINE_ENVIRONMENT_ID --output json | jq -r '.global.pdb'
```

If null is returned, the chart is using the default values and no PodDisruptionBudget resources are created.

Troubleshooting:

If pod disruptions are rejected or PDB errors appear, verify the following:

1. Verify that the PDB resources were created successfully with
   ```
   kubectl get pdb -A -l 'app.kubernetes.io/part-of=qlik-dynamic-engine'
   ```

   Look for PDB resources in the qlik-dynamic-engine and qlik-processing-env-&lt;env-id&gt; namespaces.
2. See the state of a PDB resource using
   ```
   kubectl describe pdb <pdb-name> -n <namespace>
   ```

   Verify the Disruptions Allowed, Current Healthy, and Desired Healthy fields.
3. Verify that the number of pods is enough to satisfy the PDB constraints with
   ```
   kubectl get pods -n <namespace> -l app=<service-name>
   ```

   If you have set minAvailable: 1 but only one pod exists, the PDB does not allow any disruptions. Consider adjusting the PDB constraints or increasing the number of replicas.
4. Review events related to disruption rejections with
   ```
   kubectl get events -n <namespace> --sort-by=.lastTimestamp | grep -i disruption
   ```

## What to do next

If you need to temporarily allow all disruptions for maintenance and you do not want to reduce the number of running replicas, disable the PDB using the following commands:

```
# for dynamic engine
helm upgrade dynamic-engine-$DYNAMIC_ENGINE_ID \
oci://ghcr.io/talend/helm/dynamic-engine --version $DYNAMIC_ENGINE_VERSION \
--reuse-values --set global.pdb.enabled=false

# for dynamic engine environment
helm upgrade dynamic-engine-environment-$DYNAMIC_ENGINE_ENVIRONMENT_ID \
oci://ghcr.io/talend/helm/dynamic-engine-environment --version $DYNAMIC_ENGINE_VERSION \
--reuse-values --set global.pdb.enabled=false
```

These commands disable the PDB rules without changing the existing Helm values.