Configuring Amazon EKS for Dynamic Engine deployment
Dynamic Engine can be deployed on Amazon Elastic Kubernetes Service (EKS) with EKS native storage systems.
This section explains prerequisites and storage configuration options for Dynamic Engine deployment. For general details on an Amazon EKS cluster, see Amazon EKS cluster lifecycle and configuration.
AWS-specific prerequisites
- AWS account with appropriate IAM permissions to manage:
- VPC (including creation)
- EC2 security groups
- EFS
- S3
- EKS
- IAM
- OIDC provider
- CloudFormation (required by eksctl)
- aws CLI v2
- eksctl
- kubectl
- The Kubernetes version must be compatible with Dynamic Engine. For a list of compatible Kubernetes versions, see Prerequisites to use Dynamic Engine.
- Sufficient quota to create the required resources.
Storage configuration
To set up an Amazon EKS cluster with Amazon EFS as the storage backend for Dynamic Engine, see Setting up Amazon EKS with EFS.
To set up an Amazon EKS cluster with Amazon S3 as the storage backend for Dynamic Engine, see Setting up Amazon EKS with S3.
If you use Amazon EKS Auto Mode, it is recommended to first apply the managed-mode configuration set in Managed Kubernetes operating modes. With that configuration set, configuration.persistence.enabled is false. As a result, the EFS and S3 persistence is necessary only when your Talend Management Console tasks require persistent storage.