Harden workload isolation with GKE Sandbox

This page describes how to use GKE Sandbox to protect the host kernel on your nodes when containers in the Pod execute unknown or untrusted code, or need extra isolation from the node. This page explains how you can enable GKE Sandbox and monitor your clusters when GKE Sandbox is running.

This page is for Security specialists who must isolate their workloads for additional protection from unknown or untrusted code. To learn more about common roles and example tasks that we reference in Google Cloud content, see Common GKE user roles and tasks.

Before reading this page, ensure that you're familiar with the general overview of GKE Sandbox.

Enable GKE Sandbox

GKE Sandbox is ready to use in Autopilot clusters running GKE version 1.27.4-gke.800 and later. To start deploying Autopilot workloads in a sandbox, skip to Working with GKE Sandbox.

To use GKE Sandbox in new or existing GKE Standard clusters, you must manually enable GKE Sandbox on the cluster.

For more information about GPU version releases, see GPU Model Support for details on GPU version releases.

Before you begin

Before you start, make sure that you have performed the following tasks:

  • Enable the Google Kubernetes Engine API.
    • Enable Google Kubernetes Engine API
  • To use the Google Cloud CLI for this task, install and then initialize the gcloud CLI. If you previously installed the gcloud CLI, get the latest version by running the gcloud components update command. Earlier gcloud CLI versions might not support running the commands in this document.

Enable GKE Sandbox on a new Standard cluster

The default node pool, which is created when you create a new cluster, can't use GKE Sandbox if it's the only node pool in the cluster, because GKE-managed system workloads must run separately from untrusted sandboxed workloads. To enable GKE Sandbox during cluster creation, you must add at least one extra node pool to the cluster.

Console

To view your clusters, visit the Google Kubernetes Engine menu in the Google Cloud console.

  1. In the Google Cloud console, go to the Create a Kubernetes cluster page.

    Go to Create a Kubernetes cluster

  2. Optional but recommended: From the navigation menu, under Cluster, click Features and select the following check boxes so that gVisor messages are logged:

    • Cloud Logging
    • Cloud Monitoring
    • Managed Service for Prometheus

  3. Click Add Node Pool.

  4. From the navigation menu, under Node Pools, expand the new node pool and click Nodes.

  5. Configure the following settings for the node pool:

    1. From the Image type drop-down list, select Container-Optimized OS with Containerd (cos_containerd). This is the only supported image type for GKE Sandbox.
    2. Under Machine Configuration, select a Series and Machine type.

    3. Optionally, if you're running a supported GKE version, select a GPU or TPU type. This must be one of the following GPU types:

      • nvidia-gb200: NVIDIA GB200 NVL72 (Preview)
      • nvidia-b200: NVIDIA B200 (180GB) (Preview)
      • nvidia-h200-141gb: NVIDIA H200 (141GB) (Preview)
      • nvidia-h100-mega-80gb: NVIDIA H100 Mega (80GB)
      • nvidia-h100-80gb: NVIDIA H100 (80GB)
      • nvidia-a100-80gb: NVIDIA A100 (80GB)
      • nvidia-tesla-a100: NVIDIA A100 (40GB)
      • nvidia-rtx-pro-6000: NVIDIA RTX PRO 6000 (Preview) (except for G4 machine types that have less than one GPU)
      • nvidia-l4: NVIDIA L4
      • nvidia-tesla-t4: NVIDIA T4
      For more information, see GPU model support.

      or the following TPU types:

      • v4
      • v5e
      • v5p
      • v6e
      • tpu7x

  6. From the navigation menu, under the name of the node pool you are configuring, click Security and select the Enable sandbox with gVisor checkbox.

  7. Continue to configure the cluster and node pools as needed.

  8. Click Create.

gcloud

GKE Sandbox can't be enabled for the default node pool, and it isn't possible to create additional node pools at the same time as you create a new cluster using the gcloud command. Instead, create your cluster as you normally would. Although optional, it's recommended that you enable Logging and Monitoring so that gVisor messages are logged.

Next, use the gcloud container node-pools create command, and set the -- sandbox flag to type=gvisor. The node image type must be cos_containerd for GKE Sandbox.

gcloud container node-pools create NODE_POOL_NAME \
  --cluster=CLUSTER_NAME \
  --node-version=NODE_VERSION \
  --machine-type=MACHINE_TYPE \
  --image-type=cos_containerd \
  --sandbox type=gvisor

Replace the following variables:

  • NODE_POOL_NAME: the name of your new node pool.
  • CLUSTER_NAME: the name of your cluster.
  • NODE_VERSION: the version to use for the node pool.
  • MACHINE_TYPE: the type of machine to use for the nodes.

To create a GPU node pool with GKE Sandbox, run the following command:

gcloud container node-pools create NODE_POOL_NAME \
  --cluster=CLUSTER_NAME \
  --node-version=NODE_VERSION \
  --machine-type=MACHINE_TYPE \
  --accelerator=type=GPU_TYPE,gpu-driver-version=DRIVER_VERSION \
  --image-type=cos_containerd \
  --sandbox type=gvisor

Replace the following:

  • GPU_TYPE: a supported GPU type. For details, see GKE Sandbox.

  • MACHINE_TYPE: a machine that matches the requested GPU type. For details, see Google Kubernetes Engine GPU Requirements.

  • DRIVER_VERSION: the NVIDIA driver version to install. Can be one of the following:

    • default: Install the default driver version for your GKE version.
    • latest: Install the latest available driver version for your GKE version. Available only for nodes that use Container-Optimized OS.

To create a TPU node pool with GKE Sandbox, run the following command:

gcloud container node-pools create NODE_POOL_NAME \
  --cluster=CLUSTER_NAME \
  --node-version=NODE_VERSION \
  --num-nodes=NUM_NODES \
  --tpu-topology=TPU_TOPOLOGY \
  --machine-type=MACHINE_TYPE \
  --image-type=cos_containerd \
  --sandbox type=gvisor
  • MACHINE_TYPE: a supported TPU type. For details, see GKE Sandbox.

Enable GKE Sandbox on an existing Standard cluster

You can enable GKE Sandbox on an existing Standard cluster by adding a new node pool and enabling the feature for that node pool.

Console

To create a new node pool with GKE Sandbox enabled:

  1. Go to the Google Kubernetes Engine page in the Google Cloud console.

    Go to Google Kubernetes Engine