在 GCE 上使用 TPU 的 OSS Kubernetes、TPU 的 DRA、DRANET (OSS) 和 Gemma 4

1. 總覽

本實驗室將介紹如何直接在 Google Compute Engine (GCE) 上建構自助式管理的 AI 基礎架構。您將使用 Terraform 和 kubeadm，在虛擬機器 (部分機器搭載 TPU) 上啟動未受管理的 Kubernetes 叢集，並使用開放原始碼驅動程式設定 Kubernetes 動態資源分配 (DRA)。您會使用下列項目：

• Google Compute Engine：提供運算資源，用於啟動叢集
• TPU - Google 客製化打造的加速器晶片。
• Kubernetes OSS - 手動安裝及設定 Kubernetes 的軟體
• OSS DRANET - DRA 網路驅動程式
• TPU 的 OSS DRA - 支援 TPU 的 DRA 驅動程式

如要設定環境，您需要部署多個獨立的虛擬私有雲網路，每個網路都有自己的子網路。這樣一來，您就能使用多個網路介面 (多 NIC) 佈建 VM 執行個體，將管理流量與高速 TPU 資料流量分開。

接著，如要啟用開放原始碼的動態資源分配 (DRA)，請安裝 DRA Google TPU 硬體驅動程式和 DRANET 網路驅動程式。接著，您將設定 Kubernetes DeviceClass，並編寫 ResourceClaimTemplate，以處理這些資源的動態佈建作業。

最後，您將使用 Neper 部署高效能基準測試工作負載，驗證工作節點之間的巨型封包網路資料路徑，然後執行 Python JAX 測試，驗證底層的 TPU 晶片。接著，您將部署 vLLM，透過 Hugging Face 使用完全隔離的硬體和網路 DRA 聲明，提供 Google 最先進的 Gemma 4 模型服務。

設定會使用 Terraform、gcloud 和 kubectl 的組合。

在本實驗室中，您將瞭解如何執行下列工作：

• 設定虛擬私有雲網路
• 在 GCE 上部署 3 個節點 (1 個標準節點和 2 個 TPU v6 節點)
• 啟動 Kubernetes
• 為 TPU 設定 OSS DRANET 和 DRA
• 基準成效
• 建立 DeviceClass 和 ResourceClaimTemplate
• 基準化評估網路和硬體效能
• 部署 Gemma 4：使用 vLLM 和有效的 DRA 聲明，在 TPU v6e 硬體上提供模型
• 測試與 LLM 的連線

在本實驗室中，您將建立下列模式。

圖 1

2. 設定 Google Cloud 服務

自修實驗室環境設定

1. 登入 Google Cloud 控制台，然後建立新專案或重複使用現有專案。如果沒有 Gmail 或 Google Workspace 帳戶，請先建立帳戶。

• 專案名稱是這個專案參與者的顯示名稱。這是 Google API 未使用的字元字串。你隨時可以更新。
• 專案 ID 在所有 Google Cloud 專案中都是不重複的，而且設定後即無法變更。Cloud 控制台會自動產生專屬字串，通常您不需要在意該字串為何。在大多數程式碼研究室中，您需要參照專案 ID (通常標示為 PROJECT_ID)。如果您不喜歡產生的 ID，可以產生另一個隨機 ID。你也可以嘗試使用自己的名稱，看看是否可用。完成這個步驟後就無法變更，且專案期間會維持不變。
• 請注意，部分 API 會使用第三個值，也就是「專案編號」。如要進一步瞭解這三種值，請參閱說明文件。

注意：專案 ID 在全域中是唯一的，選定後就無法再供他人使用。您是該 ID 的唯一使用者。即使專案已刪除，ID 也無法再次使用

注意：如果使用 Gmail 帳戶，可以將預設位置設為「沒有機構」。如果你使用 Google Workspace 帳戶，請選擇適合貴機構的位置。

2. 接著，您需要在 Cloud 控制台中啟用帳單，才能使用 Cloud 資源/API。完成這個程式碼研究室的費用不高，甚至可能完全免費。如要關閉資源，避免在本教學課程結束後繼續產生費用，請刪除您建立的資源或專案。Google Cloud 新使用者可參加價值$300 美元的免費試用計畫。

啟動 Cloud Shell

雖然您可以透過筆電遠端操作 Google Cloud，但在本程式碼研究室中，您將使用 Google Cloud Shell，這是可在雲端執行的指令列環境。

在 Google Cloud 控制台中，點選右上角工具列的 Cloud Shell 圖示：

佈建並連線至環境的作業需要一些時間才能完成。完成後，您應該會看到如下的內容：

這部虛擬機器搭載各種您需要的開發工具，並提供永久的 5 GB 主目錄，而且可在 Google Cloud 運作，大幅提升網路效能並強化驗證功能。您可以在瀏覽器中完成本程式碼研究室的所有作業。您不需要安裝任何軟體。

3. 使用 Terraform 設定環境

如要完成這個實驗室，您必須有權存取 TPU。使用的確切版本為 TPU v6e。

• 請按照 TPU 計畫文件啟用 TPU 配額，取得存取權。
• 使用您有 TPU 配額的區域。如需更多資訊，請參閱「在 GKE 中驗證 TPU 可用性」一文。
• 我們使用的小型部署作業需要 (2) 4 個 TPU v6e 晶片 (ct6e-standard-4t)單一區域中的 2x2 切片)。
• Hugging Face 權杖：下載 Gemma 模型權重時，需要存取權杖

注意：本實驗室會產生資源使用費用。請注意這一點，並在完成後刪除資源。

我們會建立三個自訂虛擬私有雲，其中包含防火牆規則和子網路。開啟 Cloud 控制台，然後選取要使用的專案。

1. 開啟控制台右上方的 Cloud Shell，確認 Cloud Shell 中顯示正確的專案 ID，並確認允許存取的任何提示。
2. 建立名為 oss-kube-dra, 的資料夾，然後移至該資料夾並新增一些變數。附註：請將「REGION」和「ZONE」的變數值更新為實際區域和可用區，預設使用的區域為「europe-west4」，預設使用的可用區為「europe-west4-a」。

mkdir -p oss-kube-dra && cd oss-kube-dra
export PROJECT_ID=$(gcloud config get-value project)
export REGION="europe-west4" 
export ZONE="europe-west4-a" 
echo $PROJECT_ID
echo $REGION
echo $ZONE

3. 接著新增一些設定檔。這些指令會建立下列 terraform.tfvars、variables.tf、vpc.tf 檔案。

cat << EOF > terraform.tfvars
project_id = "${PROJECT_ID}"
region     = "${REGION}"
zone       = "${ZONE}"
EOF

cat << 'EOF' > variables.tf
variable "project_id" {
  type        = string
  description = "The Google Cloud Project ID"
}

variable "region" {
  type        = string
  description = "The region to deploy the resources"
}

variable "zone" {
  type        = string
  description = "The specific zone for the VMs"
}

variable "control_plane_machine_type" {
  type        = string
  default     = "e2-standard-8"
  description = "Machine type for the Kubernetes control plane node"
}

variable "tpu_worker_machine_type" {
  type        = string
  default     = "ct6e-standard-4t"
  description = "The machine type for TPU workers (TPU v6e Trillium VM)"
}
EOF


cat << 'EOF' > vpc.tf
terraform {
  required_version = ">= 1.5.0"
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 7.32.0"
    }
  }
}

provider "google" {
  project = var.project_id
  region  = var.region
}

# 1. Primary Management VPC and Subnet
resource "google_compute_network" "primary_vpc" {
  name                    = "oss-k8s-primary-vpc"
  auto_create_subnetworks = false
  mtu                     = 1460
}

resource "google_compute_subnetwork" "primary_subnet" {
  name          = "oss-k8s-primary-subnet"
  ip_cidr_range = "10.0.0.0/24"
  region        = var.region
  network       = google_compute_network.primary_vpc.id
}

# 2. Cloud NAT Router and NAT Gateway for Primary VPC (Outbound Access)
resource "google_compute_router" "router" {
  name    = "oss-k8s-router"
  network = google_compute_network.primary_vpc.id
  region  = var.region
}

resource "google_compute_router_nat" "nat" {
  name                               = "oss-k8s-nat"
  router                             = google_compute_router.router.name
  region                             = var.region
  nat_ip_allocate_option             = "AUTO_ONLY"
  source_subnetwork_ip_ranges_to_nat = "ALL_SUBNETWORKS_ALL_IP_RANGES"
}

# 3. Firewalls for Primary VPC
resource "google_compute_firewall" "allow_internal" {
  name    = "oss-k8s-primary-allow-internal"
  network = google_compute_network.primary_vpc.id

  allow {
    protocol = "tcp"
  }
  allow {
    protocol = "udp"
  }
  allow {
    protocol = "icmp"
  }

  source_ranges = ["10.0.0.0/24"]
}

resource "google_compute_firewall" "allow_iap" {
  name    = "oss-k8s-allow-iap-ssh"
  network = google_compute_network.primary_vpc.id

  allow {
    protocol = "tcp"
    ports    = ["22"]
  }

  source_ranges = ["35.235.240.0/20"]
}

# 4. Multi-NIC TPU Networks and Subnets (With Jumbo Frames MTU 8896)
resource "google_compute_network" "tpu_vpc" {
  count                   = 2
  name                    = "oss-tpu-vpc-${count.index + 1}"
  auto_create_subnetworks = false
  mtu                     = 8896
}

resource "google_compute_subnetwork" "tpu_subnet" {
  count         = 2
  name          = "oss-tpu-vpc-${count.index + 1}-subnet"
  ip_cidr_range = "10.${count.index + 1}0.0.0/24"
  region        = var.region
  network       = google_compute_network.tpu_vpc[count.index].id
}

resource "google_compute_firewall" "tpu_allow_internal" {
  count   = 2
  name    = "oss-tpu${count.index + 1}-allow-internal"
  network = google_compute_network.tpu_vpc[count.index].id

  allow {
    protocol = "tcp"
  }
  allow {
    protocol = "udp"
  }
  allow {
    protocol = "icmp"
  }

  source_ranges = ["10.${count.index + 1}0.0.0/24"]
}
EOF

terraform.tfvars 檔案會自動擷取並插入有效的 Google Cloud 專案 ID，因此您不必在部署期間手動傳遞該 ID。

variables.tf 檔案會新增專案名稱、地區、區域、控制層和 TPU 節點的機器類型。

vpc.tf 檔案會新增相關供應商，以支援 Terraform、專案中的新自訂 VPC (含區域子網路) 和防火牆規則。

4. 確認您位於 oss-kube-dra 目錄，然後執行下列指令
   terraform init 初始化工作目錄。這是第一個步驟，會下載指定設定所需的供應商。
   terraform plan -out 會產生執行計畫，顯示 Terraform 將採取哪些動作來部署基礎架構。-out 可讓您將執行計畫儲存為具名二進位檔。您可以查看不進行任何變更會發生什麼情況。
   terraform apply 執行更新。

terraform init 
terraform plan -out=tfplan 

5. 現在請在執行 terraform apply 後執行部署作業，由於您要套用已儲存的執行計畫，系統會立即執行，不會提示您確認。(這項作業可能需要 5 到 10 分鐘)

terraform apply tfplan

6. 驗證設定。

echo -e "\n=== Verifying VPC Networks ==="
gcloud compute networks list --filter="name~oss-.*" --project=$PROJECT_ID

echo -e "\n=== Verifying Subnetworks ==="
gcloud compute networks subnets list --filter="name~oss-.*" --project=$PROJECT_ID

echo -e "\n=== Verifying Firewall Rules ==="
gcloud compute firewall-rules list --filter="name~oss-.*" --project=$PROJECT_ID

echo -e "\n=== Verifying Cloud NAT ==="
gcloud compute routers nats list --router=oss-k8s-router --router-region=$REGION --project=$PROJECT_ID

建立 VM 節點

現在，您要定義 Compute Engine 執行個體。

7. 確認您位於 oss-kube-dra 目錄，並在 Cloud Shell 中執行下列指令，寫入 nodes.tf 檔案。

cat << 'EOF' > nodes.tf
# 1. K8s Control Plane VM (No TPU)
resource "google_compute_instance" "control_plane" {
  name         = "k8s-control-plane"
  machine_type = var.control_plane_machine_type
  zone         = var.zone

  boot_disk {
    initialize_params {
      image = "projects/ubuntu-os-cloud/global/images/family/ubuntu-2204-lts"
      size  = 100
    }
  }

  network_interface {
    network    = google_compute_network.primary_vpc.id
    subnetwork = google_compute_subnetwork.primary_subnet.id
    # No public IP block keeps this node private
  }

  service_account {
    scopes = ["cloud-platform"]
  }
}

# 2. TPU Worker VMs (Multi-NIC ct6e-standard-4t instances)
resource "google_compute_instance" "tpu_workers" {
  count        = 2
  name         = "k8s-tpu-worker-${count.index + 1}"
  machine_type = var.tpu_worker_machine_type
  zone         = var.zone

  boot_disk {
    initialize_params {
      image = "projects/ubuntu-os-accelerator-images/global/images/family/ubuntu-accel-2204-amd64-tpu-v5e-v5p-v6e"
      size  = 200
    }
  }

  scheduling {
    on_host_maintenance = "TERMINATE"
    provisioning_model  = "STANDARD"
  }

  # NIC 1: Management VPC Subnet
  network_interface {
    network    = google_compute_network.primary_vpc.id
    subnetwork = google_compute_subnetwork.primary_subnet.id
  }

  # NIC 2: TPU VPC 1 Subnet
  network_interface {
    network    = google_compute_network.tpu_vpc[0].id
    subnetwork = google_compute_subnetwork.tpu_subnet[0].id
  }

  # NIC 3: TPU VPC 2 Subnet
  network_interface {
    network    = google_compute_network.tpu_vpc[1].id
    subnetwork = google_compute_subnetwork.tpu_subnet[1].id
  }

  service_account {
    scopes = ["cloud-platform"]
  }

  
  lifecycle {
    ignore_changes = [
      boot_disk[0].initialize_params[0].image,
      guest_accelerator,
      metadata
    ]
  }
}
EOF

8. 編寫新設定後，請產生新計畫並套用，以佈建執行個體。

terraform plan -out=tfplan

terraform apply tfplan

9. 驗證。

echo -e "\n=== Verifying Provisioned VM Instances ==="
gcloud compute instances list --filter="name~k8s-.*" --project=$PROJECT_ID


echo -e "\n=== Verifying Network Interfaces on Workers ==="
for i in 1 2; do
  echo -e "\n--- Interfaces for k8s-tpu-worker-${i} ---"
  gcloud compute instances describe k8s-tpu-worker-${i} \
      --zone=$ZONE \
      --project=$PROJECT_ID \
      --format="table(networkInterfaces[].network.basename(), networkInterfaces[].networkIP)"
done

完成設定後，請務必進行驗證。

4. 啟動 Kubernetes 叢集控制節點

在本節中，您將安全地連線至新建立的控制層 VM 執行個體、設定基礎作業系統、安裝容器執行階段和 Kubernetes 套件、初始化叢集，以及部署 Calico CNI，並嚴格隔離管理網路的流量。

1. 使用 GCE 的 Identity-Aware Proxy (IAP) 通道，安全連線至 k8s-control-plane 執行個體。在 Cloud Shell 終端機中執行下列指令：

gcloud compute ssh k8s-control-plane \
    --zone=$ZONE \
    --tunnel-through-iap

2. 在 k8s-control-plane VM 上建立名為 init-control-plane.sh 的指令碼，自動執行安裝和設定步驟。

cat << 'CONTROL_PLANE_EOF' > init-control-plane.sh
#!/bin/bash
# Strict error handling: fail instantly if any command exits with a non-zero status
set -e

echo "=== 1. Neutralizing Background Updates & Preparing Base OS ==="
# Prevent unattended upgrades from locking apt or breaking network configuration mid-setup
sudo systemctl stop apt-daily.timer apt-daily-upgrade.timer || true
sudo systemctl disable apt-daily.timer apt-daily-upgrade.timer || true
sudo systemctl mask apt-daily.service apt-daily-upgrade.service || true

# Turn off swap (mandatory for Kubernetes)
sudo swapoff -a
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

# Load required kernel modules
cat << 'EOT' | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOT

sudo modprobe overlay
sudo modprobe br_netfilter

# Configure sysctl requirements for Kubernetes bridging
cat << 'EOT' | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOT
sudo sysctl --system

echo "=== 2. Installing Container Runtime (Containerd) ==="
sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg bash-completion

sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor --yes -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io

echo "=== 3. Configuring Containerd with Systemd Cgroups ==="
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml >/dev/null
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml

sudo systemctl daemon-reload
sudo systemctl restart containerd
sudo systemctl enable containerd

# Validation Step: Verify runtime engine health
if ! systemctl is-active --quiet containerd; then
    echo "❌ ERROR: Containerd failed to start properly."
    exit 1
fi
echo "✅ Containerd runtime is active and healthy."

echo "=== 4. Installing Kubernetes 1.36 Binaries ==="
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.36/deb/Release.key | sudo gpg --dearmor --yes -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.36/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

# Configure Autocomplete and Aliases system-wide
kubectl completion bash | sudo tee /etc/bash_completion.d/kubectl > /dev/null
kubeadm completion bash | sudo tee /etc/bash_completion.d/kubeadm > /dev/null
if ! grep -q 'alias k=kubectl' ~/.bashrc; then
  echo 'alias k=kubectl' >> ~/.bashrc
  echo 'complete -o default -F __start_kubectl k' >> ~/.bashrc
fi

echo "=== 5. Initializing Control Plane Engine ==="
sudo kubeadm init --pod-network-cidr=192.168.0.0/16

echo "=== 6. Configuring Administrative Cluster Credentials ==="
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

# Validation Step: Verify API Server local responsiveness
echo "Waiting for local API server context..."
until kubectl cluster-info &>/dev/null; do
    sleep 2
done
echo "✅ Kubernetes API server is responding locally."

echo "=== 7. Deploying Calico Network Operator ==="
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.27.3/manifests/tigera-operator.yaml

# Validation Step: Ensure Tigera Operator CRD is fully available before applying configuration
echo "Waiting for Tigera Installation CRD to register on the API server..."
kubectl wait --for=condition=established crd/installations.operator.tigera.io --timeout=60s

echo "=== 8. Deploying Calico Custom Resources (Subnet Interlock Locked to 10.0.0.0/24) ==="
cat << 'CALICO_EOF' > custom-calico.yaml
apiVersion: operator.tigera.io/v1
kind: Installation
metadata:
  name: default
spec:
  calicoNetwork:
    nodeAddressAutodetectionV4:
      cidrs:
        - "10.0.0.0/24"
    ipPools:
    - blockSize: 26
      cidr: 192.168.0.0/16
      encapsulation: VXLANCrossSubnet
      natOutgoing: Enabled
      nodeSelector: all()
CALICO_EOF
kubectl apply -f custom-calico.yaml

# Validation Step: Confirm Calico daemon configurations are processing
echo "Waiting 10 seconds for Calico system namespaces to initialize..."
sleep 10
echo "Current Calico workload deployment status:"
kubectl get pods -n calico-system

echo "=== 9. Exporting Worker Cluster Join Token ==="
sudo kubeadm token create --print-join-command > ~/join.sh
chmod +x ~/join.sh

echo "--------------------------------------------------------"
echo "✅ CONTROL PLANE BOOTSTRAP COMPLETE!"
echo "Your cluster join command for the TPU workers is saved below:"
echo "--------------------------------------------------------"
cat ~/join.sh
CONTROL_PLANE_EOF

3. 執行指令碼。

chmod +x init-control-plane.sh
./init-control-plane.sh

4. 完成後，請進行驗證。所有功能會在幾分鐘內啟用。

kubectl get nodes
kubectl get pods -A

畫面應如下所示

NAME                STATUS   ROLES           AGE     VERSION
k8s-control-plane   Ready    control-plane   6m50s   v1.36.2
NAMESPACE         NAME                                        READY   STATUS    RESTARTS   AGE
calico-system     calico-kube-controllers-5578ff64dd-87vp2    1/1     Running   0          6m33s
calico-system     calico-node-fxzpp                           1/1     Running   0          6m33s
calico-system     calico-typha-785cbc858-rv4nz                1/1     Running   0          6m33s
calico-system     csi-node-driver-wlrhx                       2/2     Running   0          6m33s
kube-system       coredns-589f44dc88-pqfrl                    1/1     Running   0          6m42s
kube-system       coredns-589f44dc88-sdwmj                    1/1     Running   0          6m42s
kube-system       etcd-k8s-control-plane                      1/1     Running   0          6m47s
kube-system       kube-apiserver-k8s-control-plane            1/1     Running   0          6m47s
kube-system       kube-controller-manager-k8s-control-plane   1/1     Running   0          6m47s
kube-system       kube-proxy-jnm2p                            1/1     Running   0          6m42s
kube-system       kube-scheduler-k8s-control-plane            1/1     Running   0          6m47s
tigera-operator   tigera-operator-6bc8d879b5-w5mrq            1/1     Running   0          6m42s

5. 結束 ssh 連線，返回 Cloud Shell

exit

完成後驗證設定

5. 新增 TPU 工作站節點

您將從 Cloud Shell 執行指令碼，安全地連線至控制平面 VM、擷取叢集加入權杖，並同時設定 TPU 工作站節點，然後註冊至叢集。

1. 在 Cloud Shell 中執行下列指令，編寫協調指令碼：

cat << 'WORKER_BOOTSTRAP_EOF' > bootstrap-workers.sh
#!/bin/bash
# Strict error handling: fail instantly if any command exits with a non-zero status
set -e

# Fetch the join command safely from the control plane
echo "Fetching join command from Control Plane..."
JOIN_CMD=$(gcloud compute ssh k8s-control-plane --zone=$ZONE --tunnel-through-iap --command="cat ~/join.sh" 2>/dev/null)

if [ -z "$JOIN_CMD" ]; then
    echo "❌ ERROR: Failed to retrieve the join command. Ensure the control plane is reachable."
    exit 1
fi

echo "✅ Successfully retrieved join command."

# Create the setup script locally to be copied to the workers
cat << 'WORKER_INIT_EOF' > init-worker.sh
#!/bin/bash
set -e

echo "=== 1. Neutralizing Background Updates & Setting Non-Interactive Mode ==="
export DEBIAN_FRONTEND=noninteractive
sudo sed -i "s/#\$nrconf{restart} = 'i';/\$nrconf{restart} = 'a';/g" /etc/needrestart/needrestart.conf 2>/dev/null || true

# Prevent unattended upgrades from tearing down network interfaces mid-setup
sudo systemctl stop apt-daily.timer apt-daily-upgrade.timer || true
sudo systemctl disable apt-daily.timer apt-daily-upgrade.timer || true
sudo systemctl mask apt-daily.service apt-daily-upgrade.service || true

echo "=== 2. Base OS Prep ==="
# Disable swap
sudo swapoff -a
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

# Load required kernel modules
cat << 'EOT' | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOT
sudo modprobe overlay
sudo modprobe br_netfilter

# Configure bridging and IP forwarding sysctls
cat << 'EOT' | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOT
sudo sysctl --system

echo "=== 3. Installing Containerd (CRI-Only) ==="
sudo apt-get update && sudo apt-get install -yq ca-certificates curl gnupg bash-completion
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor --yes -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Install only containerd to avoid unnecessary Docker CE overhead
sudo apt-get update && sudo apt-get install -yq containerd.io
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml >/dev/null
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
sudo systemctl daemon-reload
sudo systemctl restart containerd
sudo systemctl enable containerd

# Validation: Check containerd status
if ! systemctl is-active --quiet containerd; then
    echo "❌ ERROR: Containerd failed to start."
    exit 1
fi

echo "=== 4. Installing Kubernetes 1.36 Binaries ==="
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.36/deb/Release.key | sudo gpg --dearmor --yes -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.36/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update && sudo apt-get install -yq kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
WORKER_INIT_EOF

# Append the actual join command to the script
echo "echo \"=== 5. Joining Cluster ===\"" >> init-worker.sh
echo "sudo $JOIN_CMD" >> init-worker.sh

# Push and run on both Workers concurrently
echo "Starting concurrent bootstrap on both workers..."

(
    echo "[Worker 1] Copying script..."
    gcloud compute scp init-worker.sh k8s-tpu-worker-1:~ --zone=$ZONE --tunnel-through-iap --quiet
    echo "[Worker 1] Executing script..."
    gcloud compute ssh k8s-tpu-worker-1 --zone=$ZONE --tunnel-through-iap --command="bash ~/init-worker.sh"
    echo "✅ [Worker 1] Bootstrap and Join complete!"
) &

(
    echo "[Worker 2] Copying script..."
    gcloud compute scp init-worker.sh k8s-tpu-worker-2:~ --zone=$ZONE --tunnel-through-iap --quiet
    echo "[Worker 2] Executing script..."
    gcloud compute ssh k8s-tpu-worker-2 --zone=$ZONE --tunnel-through-iap --command="bash ~/init-worker.sh"
    echo "✅ [Worker 2] Bootstrap and Join complete!"
) &

# Wait for both background processes to finish
wait

echo "--------------------------------------------------------"
echo "✅ BOTH WORKERS HAVE FINISHED PROCESSING"
echo "--------------------------------------------------------"

# Final Validation Check from Control Plane
echo "Verifying cluster node status..."
sleep 5 # Give kubelet a moment to register the nodes
gcloud compute ssh k8s-control-plane --zone=$ZONE --tunnel-through-iap --command="kubectl get nodes -o wide"
WORKER_BOOTSTRAP_EOF

2. 執行 Worker 設定。(這個程序會在背景同時執行兩項安裝作業，大約需要 3 到 5 分鐘才能完成)。

chmod +x bootstrap-workers.sh
./bootstrap-workers.sh

所有節點都新增至叢集後，您應該會看到類似的內容

To increase the performance of the tunnel, consider installing NumPy. For instructions,
please see https://cloud.google.com/iap/docs/using-tcp-forwarding#increasing_the_tcp_upload_bandwidth

NAME                STATUS     ROLES           AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION           CONTAINER-RUNTIME
k8s-control-plane   Ready      control-plane   25m   v1.36.2   10.0.0.2      <none>        Ubuntu 22.04.5 LTS   6.8.0-1064-gcp (amd64)   containerd://2.2.6
k8s-tpu-worker-1    NotReady   <none>          10s   v1.36.2   10.0.0.3      <none>        Ubuntu 22.04.5 LTS   6.8.0-1064-gcp (amd64)   containerd://2.2.6
k8s-tpu-worker-2    Ready      <none>          27s   v1.36.2   10.0.0.4      <none>        Ubuntu 22.04.5 LTS   6.8.0-1064-gcp (amd64)   containerd://2.2.6

6. 部署 OSS DRA TPU 驅動程式

在本節中，您將返回控制層，使用特定加速器拓撲詳細資料標記 TPU 工作站節點，並使用 Helm 安裝開放原始碼 Google TPU DRA 驅動程式。這個驅動程式負責探索實體 TPU v6e 晶片，並將這些晶片原生對應至 Kubernetes API。

1. 從 Cloud Shell 安全地重新連線至 k8s-control-plane VM。

gcloud compute ssh k8s-control-plane \
    --zone=$ZONE \
    --tunnel-through-iap

2. 在 k8s-control-plane SSH 工作階段中執行這些指令。使用完整標籤集 (包括確切的晶片數鍵) 標記節點

kubectl label node k8s-tpu-worker-1 \
  cloud.google.com/gke-tpu-accelerator=tpu-v6e-slice \
  cloud.google.com/gke-tpu-topology=2x2 \
  cloud.google.com/gke-tpu-dra-driver=true \
  cloud.google.com/gke-accelerator-count=4 \
  cloud.google.com/gke-tpu-count=4 \
  --overwrite

kubectl label node k8s-tpu-worker-2 \
  cloud.google.com/gke-tpu-accelerator=tpu-v6e-slice \
  cloud.google.com/gke-tpu-topology=2x2 \
  cloud.google.com/gke-tpu-dra-driver=true \
  cloud.google.com/gke-accelerator-count=4 \
  cloud.google.com/gke-tpu-count=4 \
  --overwrite

3. 使用 Helm 複製並安裝 DRA TPU 驅動程式

curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash 

git clone https://github.com/kubernetes-sigs/dra-driver-google-tpu.git ~/dra-driver-google-tpu || true
cd ~/dra-driver-google-tpu
rm -f *.pack *.tgz

helm install dra-driver-google-tpu ./deployments/helm/dra-driver-google-tpu \
  -n dra-driver-google-tpu \
  --create-namespace \
  --set 'kubeletPlugin.env[0].name=NODE_NAME' \
  --set 'kubeletPlugin.env[0].valueFrom.fieldRef.fieldPath=spec.nodeName'

cd ~

4. 驗證 DRA TPU 驅動程式設定

# Verify driver daemonset status (Pods should show as Running and Ready)
kubectl get pods -n dra-driver-google-tpu -o wide

# Verify TPU ResourceSlices are successfully published to the API server
kubectl get resourceslices

# Safely parse the ResourceSlices to show the Node Name and the number of TPU chips registered
kubectl get resourceslices -o json | jq -r '.items[] | select(.spec.driver=="tpu.google.com") | "Node: \(.spec.nodeName) | TPUs Registered: \(.spec.devices | length)"'

# Inspect driver logs to confirm the TPU hardware was initialized successfully
kubectl logs -n dra-driver-google-tpu -l app.kubernetes.io/name=dra-driver-google-tpu -c tpu-dra-plugin --tail=20

7. 部署開放原始碼 DRANET 和裝置類別

在本節中，您將返回控制層、安裝開放原始碼 DRANET 驅動程式、套用自訂篩選器修補程式來排除虛擬介面，並建立 Kubernetes DeviceClass 和 ResourceClaimTemplate，其中包含相符的 OSS 網路前置字元。

5. 從 Cloud Shell 安全地重新連線至 k8s-control-plane VM。如已連線，請略過此步驟。

gcloud compute ssh k8s-control-plane \
    --zone=$ZONE \
    --tunnel-through-iap

6. 在 k8s-control-plane SSH 工作階段中執行下列指令

# Install the core components and patch
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/dranet/refs/heads/main/install.yaml


kubectl patch daemonset dranet -n kube-system --type='json' -p='[ { "op": "add", "path": "/spec/template/spec/containers/0/args/-", "value": "-filter=!(\"dra.net/type\" in attributes) || (attributes[\"dra.net/type\"].StringValue != \"veth\" && attributes[\"dra.net/type\"].StringValue != \"vxlan\" && attributes[\"dra.net/type\"].StringValue != \"bridge\")" } ]'

# Monitor rollout readiness
kubectl rollout status daemonset/dranet -n kube-system

# Verify running components and permissions
kubectl get pods -n kube-system -l app=dranet -o wide
kubectl get clusterrole,clusterrolebinding,sa dranet -n kube-system

# Interrogate logs for driver binding confirmation
kubectl logs -n kube-system -l app=dranet --tail=20

7. 套用 DeviceClass 和 ResourceClaimTemplate

# Apply DRANET DeviceClass and BOTH ResourceClaimTemplates (Network + Hardware)
cat << 'EOF' | kubectl apply -f -
apiVersion: resource.k8s.io/v1
kind: DeviceClass
metadata:
  name: dranet
spec:
  selectors:
    - cel:
        expression: device.driver == "dra.net"
---
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: tpu-net-interfaces
  namespace: default
spec:
  spec:
    devices:
      requests:
      - name: tpu-net-interface
        exactly:
          deviceClassName: dranet
          count: 2
          selectors:
          - cel:
              expression: device.attributes["gce.dra.net"].networkName.startsWith("oss-tpu-vpc")
      config:
      - opaque:
          driver: dra.net
          parameters:
            interface:
              mtu: 8896
              gsoMaxSize: 65536
              groMaxSize: 65536
              gsoIPv4MaxSize: 65536
              groIPv4MaxSize: 65536
              disableEbpfPrograms: true
---
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: tpu-device-template
  namespace: default
spec:
  spec:
    devices:
      requests:
      - name: tpu-devices
        exactly:
          deviceClassName: tpu.google.com
          allocationMode: ExactCount
          count: 4
EOF

8. 確認範本和類別已在 Kubernetes API 中正確註冊。

# Verify ResourceSlices exist and are actively serving both drivers
kubectl get resourceslices -o custom-columns=NAME:.metadata.name,NODE:.spec.nodeName,DRIVER:.spec.driver | grep -E "dra.net|tpu.google.com"

# Verify the DRANET daemonset pods are Running across all nodes
kubectl get pods -n kube-system -l app=dranet -o wide

9. 部署 Parallel Neper StatefulSet。

cat << 'EOF' | kubectl apply -f -
---
apiVersion: v1
kind: Service
metadata:
  name: neper
spec:
  clusterIP: None
  selector:
    app: neper
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: neper
spec:
  selector:
    matchLabels:
      app: neper
  serviceName: neper
  replicas: 2
  template:
    metadata:
      labels:
        app: neper
    spec:
      initContainers:
      - name: "network-optimization-sysctls"
        image: "busybox"
        securityContext:
          privileged: true
        command:
        - sh
        - -c
        - |
          echo 5000 > /proc/sys/net/ipv4/tcp_rto_min_us
          echo 1 > /proc/sys/net/ipv4/tcp_no_metrics_save
          echo 0 > /proc/sys/net/ipv4/tcp_slow_start_after_idle
          echo 131072 > /proc/sys/net/core/optmem_max
          echo "4096 41943040 314572800" > /proc/sys/net/ipv4/tcp_rmem          
      containers:
      - name: neper
        image: ubuntu:22.04
        command:
        - /bin/bash
        - -c
        - |
          apt-get update && apt-get install -y iproute2 build-essential git jq python3-pip &&
          git clone https://github.com/google/neper.git /tmp/neper &&
          cd /tmp/neper && make &&
          cp tcp_stream /usr/local/bin/ &&
          sleep infinity
        securityContext:
          privileged: true
        resources:
          requests:
            cpu: "170"
            memory: "650Gi"
          limits:
            cpu: "170"
            memory: "650Gi"
          claims:
          - name: tpu-net-claim
          - name: tpu-hardware-claim
      resourceClaims:
      - name: tpu-net-claim
        resourceClaimTemplateName: tpu-net-interfaces
      - name: tpu-hardware-claim
        resourceClaimTemplateName: tpu-device-template
EOF

10. 驗證檢查

echo -e "\n=== Verifying StatefulSet Pod Status ==="
kubectl get pods -l app=neper -o wide

echo -e "\n=== Verifying Dynamic Resource Claims (DRCs) ==="
kubectl get resourceclaims

echo -e "\n=== Inspecting Device Claim Allocation ==="
# Using a safer JSONPath query to extract the allocated drivers and devices
kubectl get resourceclaims -o json | jq -r '.items[] | "Claim: \(.metadata.name) | Driver: \(.status.allocation.devices.results[0].driver // "Pending")"'

8. 執行測試

執行雙介面基準測試和硬體驗證套件。

第 1 階段 (網路基準測試)：等待兩個 Neper Pod (neper-0 和 neper-1) 編譯依附元件、擷取透過 DRANET 繫結的非預設多 NIC IP 位址、在 neper-1 上啟動並行 tcp_stream 伺服器、從 neper-0 產生高處理量負載，以及以每秒十億位元 (Gbps) 為單位剖析匯總處理量。

第 2 階段 (硬體驗證)：在 neper-0 中安裝 Google JAX，並透過 VFIO 在對應的 TPU 晶片上直接執行矩陣乘法 (5000x5000)，確認矽晶片運作狀態。

1. 在 k8s-control-plane 中執行下列指令，即可寫入 run_dual_neper_test.sh

cat << 'EOF' > run_dual_neper_test.sh
#!/bin/bash
set -e

SERVER_POD="neper-1"
CLIENT_POD="neper-0"

echo "================================================="
echo " PHASE 1: DUAL-INTERFACE HIGH-SPEED NETWORK TEST"
echo "================================================="
echo "=== Waiting for Pods to be Ready ==="
kubectl wait --for=condition=ready pod/$CLIENT_POD pod/$SERVER_POD --timeout=300s

echo "=== Waiting for neper compilation to finish inside Pods ==="
for POD in $SERVER_POD $CLIENT_POD; do
  until kubectl exec $POD -c neper -- sh -c 'command -v jq >/dev/null 2>&1 && command -v tcp_stream >/dev/null 2>&1'; do
    sleep 5
  done
done

echo ""
echo "=== Step 1: Extract Target IPs from $SERVER_POD ==="
# Using jq to parse the network interfaces directly from Linux JSON output
IFACE1=$(kubectl exec $SERVER_POD -c neper -- sh -c "ip -j -4 addr show | jq -r '.[] | select(.ifname != \"lo\" and .ifname != \"eth0\") | .ifname' | sed -n '1p'")
IFACE2=$(kubectl exec $SERVER_POD -c neper -- sh -c "ip -j -4 addr show | jq -r '.[] | select(.ifname != \"lo\" and .ifname != \"eth0\") | .ifname' | sed -n '2p'")

IP1=$(kubectl exec $SERVER_POD -c neper -- sh -c "ip -j -4 addr show | jq -r '.[] | select(.ifname != \"lo\" and .ifname != \"eth0\") | .addr_info[0].local' | sed -n '1p'")
IP2=$(kubectl exec $SERVER_POD -c neper -- sh -c "ip -j -4 addr show | jq -r '.[] | select(.ifname != \"lo\" and .ifname != \"eth0\") | .addr_info[0].local' | sed -n '2p'")

echo "   📍 Target IP 1 ($IFACE1): $IP1"
echo "   📍 Target IP 2 ($IFACE2): $IP2"

echo ""
echo "=== Step 2: Initialize TCP Servers on $SERVER_POD ==="
kubectl exec $SERVER_POD -c neper -- sh -c '
for i in 0 1; do
  nohup tcp_stream -C$((52279 + i)) --port=$((38339 + i)) --skip-rx-copy -rw -Z -B16384 \
    --test-length=60 --suicide-length=120 -F100 --num-threads=16 --num-flows=32 -D0 \
    --logtostderr > test${i}.log 2>&1 &
done
'
sleep 3

echo "=== Step 3: Generate Concurrent High-Throughput Load from $CLIENT_POD ==="
echo "Blasting Traffic via Interface 1 -> $IP1 ..."
kubectl exec $CLIENT_POD -c neper -- sh -c "nohup tcp_stream -C52279 --port=38339 --skip-rx-copy -rw -Z -B16384 \
  --test-length=60 --suicide-length=70 -F100 --num-threads=16 --num-flows=32 \
  --client -H $IP1 -D0 --logtostderr > test0.log 2>&1 &"

echo "Blasting Traffic via Interface 2 -> $IP2 ..."
kubectl exec $CLIENT_POD -c neper -- sh -c "nohup tcp_stream -C52280 --port=38340 --skip-rx-copy -rw -Z -B16384 \
  --test-length=60 --suicide-length=70 -F100 --num-threads=16 --num-flows=32 \
  --client -H $IP2 -D0 --logtostderr > test1.log 2>&1 &"

echo ""
echo "=== Testing in progress... Waiting 65 seconds for test completion ==="
sleep 65

echo ""
echo "=== Step 4: Evaluate Throughput Metrics ==="
RAW_BPS1=$(kubectl exec $CLIENT_POD -c neper -- grep -a "remote_throughput=" test0.log | cut -d= -f2 | tr -d '\r' || echo "0")
RAW_BPS2=$(kubectl exec $CLIENT_POD -c neper -- grep -a "remote_throughput=" test1.log | cut -d= -f2 | tr -d '\r' || echo "0")

GBPS1=$(awk -v bps="$RAW_BPS1" 'BEGIN { printf "%.2f", bps / 1000000000 }')
GBPS2=$(awk -v bps="$RAW_BPS2" 'BEGIN { printf "%.2f", bps / 1000000000 }')
TOTAL=$(awk -v b1="$RAW_BPS1" -v b2="$RAW_BPS2" 'BEGIN { printf "%.2f", (b1 + b2) / 1000000000 }')

echo "📊 --- NETWORK RESULTS ---"
echo "Interface 1 ($IFACE1) : ${GBPS1} Gbps"
echo "Interface 2 ($IFACE2) : ${GBPS2} Gbps"
echo "🔥 TOTAL AGGREGATE  : ${TOTAL} Gbps"
echo "--------------------------"


echo ""
echo "================================================="
echo " PHASE 2: TPU HARDWARE VALIDATION TEST"
echo "================================================="
echo "⏳ Installing Python and Google JAX on $CLIENT_POD (Takes ~1 minute)..."
kubectl exec $CLIENT_POD -c neper -- bash -c "apt-get update > /dev/null 2>&1 && apt-get install -y python3-pip > /dev/null 2>&1 && pip3 install jax[tpu] -f https://storage.googleapis.com/jax-releases/libtpu_releases.html > /dev/null 2>&1"

echo "🧠 Running matrix math directly on the TPU chips..."
kubectl exec $CLIENT_POD -c neper -- python3 -c "
import jax
import jax.numpy as jnp
print(f'✅ TPU Hardware Detected: {jax.device_count()} chips mapped via vfio')
print('🚀 Executing 5000x5000 Matrix Multiplication on TPU silicon...')
x = jnp.ones((5000, 5000))
y = jnp.dot(x, x)
print('✅ Success! The TPU driver is fully operational and executing math.')
"
EOF

chmod +x run_dual_neper_test.sh

2. 執行測試。這項作業需要 2 分鐘才能完成。

./run_dual_neper_test.sh

完成後，終端機輸出內容會顯示經過驗證的高速網路指標和 TPU 矩陣數學執行作業

=== Step 4: Evaluate Throughput Metrics ===
📊 --- NETWORK RESULTS ---
Interface 1 (ens9)  : 157.51 Gbps
Interface 2 (ens10) : 167.04 Gbps
🔥 TOTAL AGGREGATE  : 324.55 Gbps
--------------------------

=================================================
 PHASE 2: TPU HARDWARE VALIDATION TEST
=================================================
⏳ Installing Python and Google JAX on neper-0 (Takes ~1 minute)...
🧠 Running matrix math directly on the TPU chips...
✅ TPU Hardware Detected: 4 chips mapped via vfio
🚀 Executing 5000x5000 Matrix Multiplication on TPU silicon...
✅ Success! The TPU driver is fully operational and executing math.

9. 在叢集上部署 Gemma 4

在本節中，您將設定安全的 Hugging Face API 憑證做為 Kubernetes 密鑰、部署 vLLM 推論引擎 (同時使用動態資源分配 (DRA) 網路和硬體聲明)，並對 Google 的 Gemma 4 模型執行端對端測試查詢。

請確認您已在 k8s-control-plane 登入安全的 SSH 工作階段：

1. 從 Cloud Shell 安全地重新連線至控制層 VM。如果已連線，請略過這個步驟。

gcloud compute ssh k8s-control-plane \
    --zone=$ZONE \
    --tunnel-through-iap

2. 清理先前的部署作業

# 1. Delete the StatefulSet to stop the benchmarking pods
kubectl delete statefulset neper

# 2. Wait for the pods to terminate fully and release the claims
kubectl wait --for=delete pod/neper-0 pod/neper-1 --timeout=60s

3. 儲存 Hugging Face 存取權杖。將 <YOUR_ACTUAL_HUGGING_FACE_TOKEN> 替換成您的權杖。

export HF_TOKEN="<YOUR_ACTUAL_HUGGING_FACE_TOKEN>"

4. 建立密鑰

kubectl create secret generic hf-token --from-literal=token="${HF_TOKEN}"

5. 這份資訊清單會排定 vLLM 的單一副本，在 4 個晶片的原始 TPU VM 上執行。這項功能會使用 Kubernetes DRA 標準，掛接自訂網路聲明 (tpu-net-claim) 和硬體聲明 (tpu-hardware-claim)，安全存取原始 TPU 硬體，不必使用不安全的主機磁碟區掛接。最後，它會透過通訊埠 8080 公開與 OpenAI 相容的 API 伺服器。執行下列指令來建立檔案：

cat << 'EOF' > gemma-inference.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: vllm-gemma-4
  labels:
    app: gemma-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: gemma-server
  template:
    metadata:
      labels:
        app: gemma-server
    spec:
      hostIPC: true
      containers:
      - name: vllm-tpu
        image: vllm/vllm-tpu:latest
        securityContext:
          privileged: true
        env:
        - name: HF_TOKEN
          valueFrom:
            secretKeyRef:
              name: hf-token
              key: token
        - name: JAX_PLATFORMS
          value: "tpu,cpu"
        - name: TPU_ACCELERATOR_TYPE
          value: "v6e-4"
        - name: TPU_WORKER_HOSTNAMES
          value: "127.0.0.1"
        - name: TPU_WORKER_ID
          value: "0"
        - name: LIBTPU_INIT_ARGS
          value: "--noenable_tpunetd_client"
        - name: BARE_METAL_MODE
          value: "true"
        - name: BYPASS_VBAR_CONTROL_SERVICE
          value: "1"
        - name: TPU_SKIP_MDS_QUERY
          value: "1"
        - name: TPU_DEFAULT_NETWORK_TYPE
          value: "loopback"
        - name: CHIPS_PER_HOST_BOUNDS
          value: "2,2,1"
        - name: HOST_BOUNDS
          value: "1,1,1"
        - name: ALT
          value: "false,false,false"
        - name: WRAP
          value: "false,false,false"
        command:
        - bash
        - -c
        - |
          export PYTHONUNBUFFERED=1
          sysctl -w net.ipv6.conf.all.disable_ipv6=0
          sysctl -w net.ipv6.conf.default.disable_ipv6=0
          sysctl -w net.ipv6.conf.lo.disable_ipv6=0
          ip link set lo up || true
          
          exec python3 -m vllm.entrypoints.openai.api_server \
            --model google/gemma-4-E4B-it \
            --tensor-parallel-size 4 \
            --trust-remote-code \
            --max-model-len 8192 \
            --max-num-batched-tokens 4096 \
            --host 0.0.0.0 \
            --port 8080
        ports:
        - containerPort: 8080
        resources:
          requests:
            cpu: "170"
            memory: "650Gi"
          limits:
            cpu: "170"
            memory: "650Gi"
          claims:
          - name: tpu-net-claim
          - name: tpu-hardware-claim
        volumeMounts:
        - name: dshm
          mountPath: /dev/shm
      volumes:
      - name: dshm
        emptyDir:
          medium: Memory
      resourceClaims:
      - name: tpu-net-claim
        resourceClaimTemplateName: tpu-net-interfaces
      - name: tpu-hardware-claim
        resourceClaimTemplateName: tpu-device-template
---
apiVersion: v1
kind: Service
metadata:
  name: vllm-gemma-service
spec:
  selector:
    app: gemma-server
  ports:
  - protocol: TCP
    port: 8080
    targetPort: 8080
  type: ClusterIP
EOF

6. 部署推論工作負載

kubectl apply -f gemma-inference.yaml

7. 確認部署狀態。這項設定必須下載模型並載入 vLLM. This，可能需要 10 - 25 minutes.

kubectl get pods -l app=gemma-server

kubectl describe pods -l app=gemma-server

您也可以觀看容器的記錄檔，瞭解程序。按下 CTRL+C 即可退出記錄檢視畫面。

kubectl logs -l app=gemma-server -f

看到以下幾行時，表示引擎已完全初始化：

(APIServer pid=1) INFO: Started server process [1]

(APIServer pid=1) INFO: Waiting for application startup.

(APIServer pid=1) INFO: Application startup complete。

按 CTRL+C 退出記錄串流，再繼續操作。

8. 驗證介面附件。檢查繫結至容器內的網路介面

kubectl exec deployment/vllm-gemma-4 -c vllm-tpu -- ls /sys/class/net
kubectl exec deployment/vllm-gemma-4 -c vllm-tpu -- cat /proc/net/fib_trie | grep -B 1 "32 host"

應注意的事項：您應該會看到 ens9 和 ens10 (或類似的 ensX 名稱)，以及標準 CNI 介面 (eth0) 和回送 (lo)。這些代表實體 GCE 主機 PCI 網路介面，由開放原始碼 DRANET 驅動程式使用 systemd 的可預測插槽命名慣例，動態繫結至 Pod 內。

kubectl exec deployment/vllm-gemma-4 -c vllm-tpu -- ls /sys/class/net
ens10
ens9
eth0
Lo

kubectl exec deployment/vllm-gemma-4 -c vllm-tpu -- cat /proc/net/fib_trie | grep -B 1 "32 host"

              |-- 10.10.0.3
                 /32 host LOCAL
--
              |-- 10.20.0.3
                 /32 host LOCAL
--
           |-- 127.0.0.1
              /32 host LOCAL
--
     |-- 192.168.238.67
        /32 host LOCAL
--
              |-- 10.10.0.3
                 /32 host LOCAL
--
              |-- 10.20.0.3
                 /32 host LOCAL
--
           |-- 127.0.0.1
              /32 host LOCAL
--
     |-- 192.168.238.67
        /32 host LOCAL

10. 測試 LLM

介面驗證完成後，請在叢集中啟動輕量級測試容器，針對 Gemma 4 傳送串流推論要求。

1. 在 k8s-control-plane 工作階段中執行下列指令，啟動互動式用戶端：

kubectl run gemma-chat --rm -i --tty --image=alpine --restart=Never -- sh -c '
  # 1. Silently install curl and jq
  apk add --no-cache curl jq > /dev/null

  echo -e "\n========================================================"
  echo -e "💬 Welcome to the Gemma 4 Real-Time CLI Chat client!"
  echo -e "========================================================"
  echo -e "   Type your prompt below. Type '\''exit'\'' or '\''quit'\'' to end."
  echo -e "========================================================\n"

  while true; do
    # Read user input
    echo -n -e "👤 \033[1;34mYou:\033[0m "
    read -r USER_INPUT
    
    # Handle exit conditions
    if [ "$USER_INPUT" = "exit" ] || [ "$USER_INPUT" = "quit" ] || [ -z "$USER_INPUT" ]; then
      echo -e "\n👋 Goodbye!"
      break
    fi

    echo -n -e "🤖 \033[1;32mGemma:\033[0m "

    # Use jq to safely escape double quotes and special characters in user input
    JSON_PAYLOAD=$(jq -n --arg msg "$USER_INPUT" '\''{
      model: "google/gemma-4-E4B-it",
      messages: [{role: "user", content: $msg}],
      temperature: 0.7,
      stream: true
    }'\'')

    # Stream the tokens in real-time with a typewriter effect
    curl -s -X POST http://vllm-gemma-service:8080/v1/chat/completions \
      -H "Content-Type: application/json" \
      -d "$JSON_PAYLOAD" | while read -r line; do
        # Extract SSE data streams
        if echo "$line" | grep -q "data:"; then
          DATA_CLEAN=$(echo "$line" | sed "s/^data: //" | tr -d "\r")
          if [ "$DATA_CLEAN" != "[DONE]" ] && [ -n "$DATA_CLEAN" ]; then
            # Parse and print only the token content
            TOKEN=$(echo "$DATA_CLEAN" | jq -r ".choices[0].delta.content // empty" 2>/dev/null)
            echo -n "$TOKEN"
          fi
        fi
      done
    echo -e "\n"
  done
'

Interactive chat

注意：這個模型會根據訓練內容回覆，可能不瞭解目前的問題。輸入「exit」即可結束互動式對話

11. 清除

注意：為避免產生不必要的費用，請務必清除環境

首先，請從叢集中刪除所有工作負載、密鑰和設定。

如果您仍登入k8s-control-plane安全殼層 (SSH) 工作階段，請直接執行下列指令。(如果已結束，請先重新透過 SSH 登入)：

1. 從 Cloud Shell 安全地重新連線至控制層 VM。如果已連線至這個 VM，請略過這個步驟。

gcloud compute ssh k8s-control-plane \
    --zone=$ZONE \
    --tunnel-through-iap

2. 清除 Kubernetes 資源

# 1. Delete the Gemma 4 deployment and service
kubectl delete -f gemma-inference.yaml --ignore-not-found=true

# 2. Delete the Hugging Face access secret
kubectl delete secret hf-token --ignore-not-found=true

# 3. Delete the open-source DRANET specs and drivers
kubectl delete deviceclass dranet --ignore-not-found=true
kubectl delete resourceclaimtemplate tpu-net-interfaces --ignore-not-found=true
kubectl delete -f https://raw.githubusercontent.com/kubernetes-sigs/dranet/refs/heads/main/install.yaml --ignore-not-found=true

# 4. Uninstall the OSS TPU Hardware Driver
helm uninstall dra-driver-google-tpu -n dra-driver-google-tpu --wait || true

3. 現在請輸入 exit，返回儲存 Terraform 檔案的現用 Cloud Shell 目錄，然後刪除所有節點、虛擬私有雲網路和防火牆規則。

# 1. Create the teardown script
cat << 'EOF' > teardown.sh
#!/bin/bash

# The specific networks defined in your Terraform vpc.tf
NETWORKS=(
  "oss-k8s-primary-vpc"
  "oss-tpu-vpc-1"
  "oss-tpu-vpc-2"
)

echo "=== Hunting down and deleting ALL firewall rules for OSS networks ==="

for NETWORK in "${NETWORKS[@]}"; do
    echo "Searching for firewall rules attached to network: $NETWORK..."
    
    # Query GCP for any firewall rule tied to this specific network
    STUCK_RULES=$(gcloud compute firewall-rules list \
        --filter="network:($NETWORK)" \
        --format="value(name)" | tr '\n' ' ')
    
    # Check if the string is not empty and contains more than just whitespace
    if [ -n "$STUCK_RULES" ] && [ "$STUCK_RULES" != " " ]; then
        echo "🔥 Found rules holding $NETWORK hostage: $STUCK_RULES"
        echo "Deleting them now..."
        gcloud compute firewall-rules delete $STUCK_RULES --quiet
    else
        echo "✅ No firewall rules found for $NETWORK."
    fi
done

# Fallback: Explicitly delete the named rules from your Terraform file 
# just in case the dynamic filter missed them due to caching delays
echo "=== Running fallback deletion for explicitly named Terraform rules ==="
gcloud compute firewall-rules delete \
    oss-k8s-primary-allow-internal \
    oss-k8s-allow-iap-ssh \
    oss-tpu1-allow-internal \
    oss-tpu2-allow-internal \
    --quiet 2>/dev/null || true

echo "--------------------------------------------------------"
echo "✅ Firewall cleanup complete!"
echo "Your networks are now stripped of firewalls and ready to be deleted."
echo "--------------------------------------------------------"

echo "=== Destroying Infrastructure ==="
cd ~/oss-kube-dra || exit
terraform destroy -auto-approve

echo "--------------------------------------------------------"
echo "✅ Infrastructure successfully destroyed!"
echo "--------------------------------------------------------"
EOF

# 2. Make the script executable and run it
chmod +x teardown.sh
./teardown.sh

4. 刪除 Terraform 資料夾 oss-kube-dra

cd
rm -r oss-kube-dra

12. 恭喜

您已成功在 Google Compute Engine (GCE) VM 執行個體上，直接佈建、啟動及驗證高效能的自管 Kubernetes AI 基礎架構。

您現在已深入瞭解 Kubernetes 如何使用動態資源分配 (DRA) 協調原始 TPU 加速器、繫結高速多 NIC 主機拓撲，以及提供尖端大型語言模型。

後續步驟/瞭解詳情

您可以進一步瞭解 GKE 網路

挑戰下一個實驗室

繼續完成 Google Cloud 任務，或查看下列其他 Google Cloud 實驗室：

• 建構多叢集 GKE 推論閘道，搭配 TPU、Cloud Storage FUSE 和代管 DRANET
• AI 基礎架構：網路技術
• 探索生成式 AI - Vertex AI
• Cloud 網路導覽