Configuring Job size limits for Dynamic Engine environments

To protect your Dynamic Engine environments from Denial of Service (DoS) attacks, you can set limits on Job artifact sizes during deployment and execution. Any Job artifact that exceeds these limits is automatically deleted.

Customize Job size limits by creating a custom values file and applying it to your Dynamic Engine environment.

Procedure

Create a custom values file for your Dynamic Engine environment with the required Job size limits to protect your system from DoS attacks.

Example

cat <<EOF > <environment-id>-custom-job-limits-values.yaml
configuration:
  jobLimits: 
    maxZipNameLength: 240
    maxZippedEntries: 2048
    maxUnzippedSize: 1000000000
    maxUnzippedFolderNameLength: 240
    maxUnzippedFileNameLength: 240
    maxZipDepth: 64
EOF

Apply the custom values file to your Dynamic Engine environment using Helm.

helm install dyn-engine-env-<environment-id> -f <environment-id>-values.yaml  \
 -f <environment-id>-custom-job-limits-values.yaml \
 oci://ghcr.io/talend/helm/dynamic-engine-environment \
 --version <engine-version>

Tip: When changes are needed, you can update the content of the <environment-id>-custom-job-limits-values.yaml file and run helm upgrade to apply the changes:

helm upgrade dyn-engine-env-<environment-id> -f <environment-id>-values.yaml  \
-f <environment-id>-custom-job-limits-values.yaml \
oci://ghcr.io/talend/helm/dynamic-engine-environment \
--version <engine-version>

Results

Verify that the jobLimits property appears in the ConfigMap called global-configuration for your Dynamic Engine environment:

"jobLimits": {
  "maxZipNameLength": 240,
  "maxZippedEntries": 2048,
  "maxUnzippedSize": 1000000000,
  "maxUnzippedFolderNameLength": 240,
  "maxUnzippedFileNameLength": 240,
  "maxZipDepth": 64
}

These fields specify the following size limits:

maxZipNameLength: Maximum length of the zip file name.
maxZippedEntries: Maximum number of files and directories in the zip file.
maxUnzippedSize: Maximum size of the uncompressed zip file in bytes.
maxUnzippedFolderNameLength: Maximum length of any folder name after unzipping.
maxUnzippedFileNameLength: Maximum length of any file name after unzipping.
maxZipDepth: Maximum nesting level of directories in each Job artifact zip file.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – please let us know!

Leave your feedback here