Configuring Google Kubernetes Engine (GKE) for Dynamic Engine deployment

GKE-specific prerequisites

• Basic GKE knowledge.
• The Google Cloud CLI installed on your machine to interact with Google Cloud and GKE. For more information, see the Google Cloud SDK installation guide.
• The gke-gcloud-auth-plugin plugin installed and authenticated to your Google Cloud project. For more information, see GKE access for kubectl.
• The Kubernetes version must be compatible with Dynamic Engine. For a list of compatible Kubernetes versions, see Prerequisites to use Dynamic Engine.

Cluster configuration

Create or update your GKE cluster by using one of the following paths:

• Standard GKE cluster:

  gcloud container clusters create $CLUSTER_NAME \
    --addons=GcpFilestoreCsiDriver \
    --cluster-version=1.30 \
    --location=us-central1-a \
    --machine-type=e2-standard-4 \
    --num-nodes=3

• Existing standard GKE cluster:

  gcloud container clusters update $CLUSTER_NAME \
    --update-addons=GcpFilestoreCsiDriver=ENABLED \
    --region=$REGION

• GKE Autopilot cluster: create the cluster with Google Cloud Autopilot workflows, then continue with the managed-mode bundle described in Managed Kubernetes operating modes.

The standard cluster commands enable GcpFilestoreCsiDriver and use a machine type with at least 4 vCPU, for example, e2-standard-4.

Cluster access

Configure access to your GKE cluster by generating and setting up your local kubeconfig credentials. For more information, see GKE access for kubectl.

To install the plugin, run:

gcloud components install gke-gcloud-auth-plugin

To authenticate with Google Cloud and set your project, run:

gcloud auth login
gcloud config set project $PROJECT_ID

Replace $PROJECT_ID with your Google Cloud project ID.

To generate your kubeconfig, run:

gcloud container clusters get-credentials $CLUSTER_NAME --region $REGION --project $PROJECT_ID

Then verify your current Kubernetes context:

kubectl config get-contexts

If the current context is not the intended one, set it:

kubectl config use-context $GKE_CONTEXT

$GKE_CONTEXT is usually gke_$PROJECT_ID_$REGION_$CLUSTER_NAME. Replace the variables with your actual values.

Ensure all system pods are running in your GKE cluster. You can run this command to check the status:

kubectl get pods -n kube-system

The Google Cloud console also provides the status information.

Storage configuration

To configure GKE storage for Talend Management Console, see Setting up GKE with Filestore storage for Dynamic Engine.

If you use GKE Autopilot, it is recommended to first apply the managed-mode configuration set in Managed Kubernetes operating modes. With that configuration set, configuration.persistence.enabled is false. As a result, Filestore persistence is necessary only when your Talend Management Console tasks require persistent storage.