# Setting up Amazon EKS with EFS

Before deploying Dynamic Engine and its environments, set up an Amazon EKS cluster with Amazon Elastic File System (EFS) as the shared storage backend.

## Before you begin

* An AWS account with the required IAM permissions. For the full list, see [Configuring Amazon EKS for Dynamic Engine deployment](https://help.qlik.com/talend/en-US/dynamic-engine-configuration-guide/Cloud/deploying-dynamic-engine-on-aws-eks "Dynamic Engine can be deployed on Amazon Elastic Kubernetes Service (EKS) with EKS native storage systems.").

## About this task

Information noteImportant: For Amazon EKS Auto Mode, it is recommended to first apply the managed-mode configuration set in [Managed Kubernetes operating modes](https://help.qlik.com/talend/en-US/dynamic-engine-configuration-guide/Cloud/prerequisites-to-use-dynamic-engines).

This procedure creates a new EKS cluster in a dedicated virtual private cloud (an Amazon VPC) and configures EFS mount targets across Availability Zones and installs the EFS CSI (Container Storage Interface) driver. The procedure also creates a storage class that references your EFS filesystem and uses access points to dynamically provision persistent volumes.

## Procedure

1. Set environment variables for your AWS EKS and EFS deployment:
   ### Example
   ```
   export AWS_REGION=<your-aws-region>
   export EKS_CLUSTER_NAME=<your-eks-cluster-name>
   export EFS_NAME=<your-efs-filesystem-name>
   ```
2. Create the EKS cluster configuration file to deploy an EKS cluster with infrastructure optimized for the Dynamic Engine environment:
   ```
   cat <<EOF > eks-config.yaml
   apiVersion: eksctl.io/v1alpha5
   kind: ClusterConfig

   metadata:
     name: ${EKS_CLUSTER_NAME}
     region: ${AWS_REGION}
     version: "1.34"

   addonsConfig:
     autoApplyPodIdentityAssociations: true

   addons:
     - name: eks-pod-identity-agent
     - name: aws-efs-csi-driver
       
   vpc:
     cidr: 10.10.0.0/16
     autoAllocateIPv6: false
     hostnameType: resource-name
     clusterEndpoints:
       publicAccess: true
       privateAccess: true
     nat:
       gateway: HighlyAvailable

   managedNodeGroups:
     - name: ng-amd64
       amiFamily: AmazonLinux2023 # (default) or Bottlerocket
       minSize: 2
       maxSize: 4
       desiredCapacity: 2
       instanceSelector:
         vCPUs: 4
         memory: 16GiB
         cpuArchitecture: amd64
       privateNetworking: true
       
       
   EOF
   ```

   This configuration:
   * Specifies the region and the name of the EKS cluster and Kubernetes version to be used (1.34 in this example).
   * Installs two addons that Dynamic Engine requires to run on EFS: aws-efs-csi-driver, which securely mounts EFS volumes onto pods, and eks-pod-identity-agent, which lets Dynamic Engine pods assume the IAM role needed to access EFS without hardcoded credentials.
   * Creates a VPC and places all nodes in private subnets (privateNetworking: true), which is required by Dynamic Engine. Nodes reach the internet only through NAT gateways to pull Dynamic Engine container images. The cluster endpoint remains accessible from both your local machine and within the VPC.
   * Provisions nodes sized for Dynamic Engine workloads. Each node has 4 vCPUs and 16 GiB of RAM. The AMD64 architecture (cpuArchitecture: amd64) is required because Dynamic Engine container images are built for AMD64 only.
3. Create the EKS cluster, link your local kubectl tool to the EKS cluster, and verify the cluster's accessibility:
   ```
   eksctl create cluster -f eks-config.yaml
   aws eks update-kubeconfig --region "$AWS_REGION" --name "$EKS_CLUSTER_NAME"
   kubectl get nodes
   ```

   Wait for all nodes to reach the Ready status before proceeding.
4. Run this script to retrieve the VPC ID and the subnet IDs:
   ```
   VPC_ID=$(aws eks describe-cluster \
     --region "$AWS_REGION" \
     --name ${EKS_CLUSTER_NAME} \
     --output text \
     --query "cluster.resourcesVpcConfig.vpcId")
   echo "VPC_ID=$VPC_ID"

   SUBNET_IDS_PRIVATE=$(aws ec2 describe-subnets \
     --region "$AWS_REGION" \
     --filters Name=vpc-id,Values="$VPC_ID" Name=tag:kubernetes.io/role/internal-elb,Values=1 \
     --query "Subnets[*].SubnetId" \
     --output text | sed 's/\t/ /g')
   echo "SUBNET_IDS_PRIVATE=$SUBNET_IDS_PRIVATE"
   ```
5. Create an EFS security group accessible from the EKS nodes.
   1. Get the ID of the EKS node security group:
      ```
      NODE_SG_ID=$(aws eks describe-cluster \
         --region "$AWS_REGION" \
         --name "$EKS_CLUSTER_NAME" \
         --query "cluster.resourcesVpcConfig.clusterSecurityGroupId" \
         --output text)
      echo "NODE_SG_ID=$NODE_SG_ID"
      ```
   2. Create EFS security group.
      ```
      EFS_SG_ID=$(aws ec2 create-security-group \
        --region "$AWS_REGION" \
        --group-name "${EKS_CLUSTER_NAME}-efs-sg" \
        --description "EFS access from EKS" \
        --vpc-id "$VPC_ID" \
        --query "GroupId" \
        --output text)
      ```
   3. Grant the node security group access to the EFS security group.
      ```
      aws ec2 authorize-security-group-ingress \
        --region "$AWS_REGION" \
        --group-id "$EFS_SG_ID" \
        --protocol tcp \
        --port 2049 \
        --source-group "$NODE_SG_ID"
      ```
6. Create the EFS filesystem:
   ```
   EFS_ID=$(aws efs create-file-system \
     --region "$AWS_REGION" \
     --creation-token "${EKS_CLUSTER_NAME}-efs" \
     --encrypted \
     --tags Key=Name,Value="${EFS_NAME}" \
     --query "FileSystemId" \
     --output text)

   echo "EFS_ID=$EFS_ID"
   ```
7. Create mount targets in each private subnet:
   ```
   for subnet in $(echo "$SUBNET_IDS_PRIVATE" | tr ' ' '\n'); do
     aws efs create-mount-target \
     --region "$AWS_REGION" \
     --file-system-id "$EFS_ID" \
     --subnet-id "$subnet" \
     --security-groups "$EFS_SG_ID"
   done
   ```

   Wait for mount targets to become available. Verify status with:
   ```
   aws efs describe-mount-targets \
     --region "$AWS_REGION" \
     --file-system-id "$EFS_ID"
   ```

   The mount targets in each subnet allow the nodes in that subnet to reach EFS.
8. Create a storage class that enables Dynamic Engine to dynamically provision EFS volumes:
   1. Create the configuration file for the storage class.
      ## Example
      ```
      cat <<EOF > efs-storageclass.yaml
      apiVersion: storage.k8s.io/v1
      kind: StorageClass
      metadata:
        name: dyn-engine
      provisioner: efs.csi.aws.com
      parameters:
        provisioningMode: efs-ap
        fileSystemId: $EFS_ID
        directoryPerms: "700"
        gidRangeStart: "1000"
        gidRangeEnd: "65534"  
      mountOptions:
        - tls
      reclaimPolicy: Delete
      volumeBindingMode: Immediate

      EOF
      ```
      * provisioningMode: efs-ap: Enables EFS Access Point mode. Each PersistentVolumeClaim is backed by a dedicated EFS Access Point, which enforces POSIX ownership and isolates directory access per volume.
      * fileSystemId: ${EFS\_ID}: This value must match the EFS filesystem ID created in step [6](https://help.qlik.com/talend/en-US/dynamic-engine-configuration-guide/Cloud/install-dynamic-engine-eks-with-efs#efs-filesystem-creation).
      * gidRangeStart: "1000" and gidRangeEnd: "65534" — This group ID range controls which POSIX groups are allowed to access EFS volumes. Dynamic Engine runs with group ID 61000 by default, which must fall within this range (1000-65534). If the group ID falls outside this range, pods receive permission-denied errors when accessing EFS volumes.

        Information noteImportant: If pods fail to mount EFS volumes with permission denied errors, verify the Dynamic Engine pod's group ID (61000) is within the gidRange settings. See [Troubleshooting permission errors when accessing EFS volumes in pods on Amazon EKS](https://help.qlik.com/talend/en-US/dynamic-engine-configuration-guide/Cloud/troubleshooting-eks-efs-access-permissions "Resolve permission-denied errors when pods fail to read or write to mounted EFS volumes on Amazon EKS.") for detailed diagnostic steps.

      For further information on a Kubernetes storage class, see [AWS EFS](https://kubernetes.io/docs/concepts/storage/storage-classes/#aws-efs).
   2. Create the storage class:
      ```
      kubectl apply -f efs-storageclass.yaml
      ```
9. Test the storage class.
   1. Create a PVC and a pod for this test.
      ## Example

      The PVC requests storage with the dyn-engine storage class and the pod mounts the volume before writing test data to it.
      ```
      cat <<EOF > efs-pvc-pod.yaml
      ---
      apiVersion: v1
      kind: PersistentVolumeClaim
      metadata:
        name: efs-pvc
      spec:
        accessModes:
          - ReadWriteMany
        storageClassName: dyn-engine
        resources:
          requests:
            storage: 100Gi
      ---
      apiVersion: v1
      kind: Pod
      metadata:
        name: efs-app
      spec:
        securityContext:
          fsGroup: 61000
          runAsNonRoot: true
          runAsUser: 61000
          runAsGroup: 61000
          seccompProfile:
            type: RuntimeDefault
        containers:
          - name: app
            image: ghcr.io/talend/kube-base:5.3.0
            command: ["/bin/sh"]
            args: ["-c", "while true; do echo $(date -u) >> /data/out; sleep 5; done"]
            securityContext:
              allowPrivilegeEscalation: false
            volumeMounts:
              - name: persistent-storage
                mountPath: /data
        volumes:
          - name: persistent-storage
            persistentVolumeClaim:
              claimName: efs-pvc
      ```
   2. Apply the file:
      ```
      kubectl apply -f efs-pvc-pod.yaml
      ```

   The /data/out file is created and contains UTC timestamps.

## Results

Your EKS cluster and its EFS storage class are ready for Dynamic Engine and its environments.

## What to do next

Deploy Dynamic Engine and its environments to the EKS cluster you configured in this procedure, and specify the storage class you created in this procedure.

For detailed instructions, see [Provisioning a storage class dedicated to Dynamic Engine environment services](https://help.qlik.com/talend/en-US/dynamic-engine-configuration-guide/Cloud/deploy-dynamic-engine-with-specific-storageclass "Configure a specific storage class for Dynamic Engine environment services instead of using the default Kubernetes storage class."). These instructions apply to all supported platforms, including Amazon EKS, Google GKE, and on-premises Kubernetes.

Information noteTip:

To clean up the test infrastructure, delete the resources in this order:

1. Delete the EKS cluster:
   ```
   eksctl delete cluster --name "$EKS_CLUSTER_NAME" --region "$AWS_REGION"
   ```
2. Delete the EFS mount targets before deleting the file system and its security group:
   ```
   MT_IDS=$(aws efs describe-mount-targets \
     --region "$AWS_REGION" \
     --file-system-id "$EFS_ID" \
     --query "MountTargets[].MountTargetId" \
     --output text)
   for mt in $(echo "$MT_IDS" | tr ' ' '\n'); do
     aws efs delete-mount-target --region "$AWS_REGION" --mount-target-id "$mt"
   done
   aws efs delete-file-system --region "$AWS_REGION" --file-system-id "$EFS_ID"
   aws ec2 delete-security-group --region "$AWS_REGION" --group-id "$EFS_SG_ID"
   ```

This information is provided for your convenience and is subject to changes on the Amazon side. See Amazon's documentation for details on EKS and EFS deletion.

### In this section

* [Troubleshooting Dynamic Engine on Amazon EKS with Amazon EFS](https://help.qlik.com/talend/en-US/dynamic-engine-configuration-guide/Cloud/troubleshooting-eks-efs-deployment)