Gemini CLI on GCE with a Private Service Connect endpoint

1. Overview

You can use Gemini CLI in both your local and virtual environments. Gemini CLI is an open-source AI agent that brings the power of Gemini directly into your terminal. From a networking point of view when you use Gemini CLI it will make a call to the Gemini APi via the API IP address which is publicly accessible.

Now what happens if you want to use Gemini CLI on a Google Compute Engine machine but you want to connect to the API privately? In this codelabs you will see how to configure Private Service Connect endpoint to Google APIs, to route the traffic to an internal IP address you specify.

The configurations will be a combination of Terraform, gcloud and console.

In this lab you will learn how to perform the following task:

• Set up VM instance and Cloud NAT
• Install Gemini CLI and authenticate
• Configure Private Service Connect endpoint to connect to Googleapis
• Verify connectivity path to *.googleais
• Configure manual DNS entries

In this lab, you're going to be creating the following pattern.

Figure1.

2. Google Cloud services setup

Self-paced environment setup

1. Sign-in to the Google Cloud Console and create a new project or reuse an existing one. If you don't already have a Gmail or Google Workspace account, you must create one.

• The Project name is the display name for this project's participants. It is a character string not used by Google APIs. You can always update it.
• The Project ID is unique across all Google Cloud projects and is immutable (cannot be changed after it has been set). The Cloud Console auto-generates a unique string; usually you don't care what it is. In most codelabs, you'll need to reference your Project ID (typically identified as PROJECT_ID). If you don't like the generated ID, you might generate another random one. Alternatively, you can try your own, and see if it's available. It can't be changed after this step and remains for the duration of the project.
• For your information, there is a third value, a Project Number, which some APIs use. Learn more about all three of these values in the documentation.

Caution: A project ID is globally unique and can't be used by anyone else after you've selected it. You are the only user of that ID. Even if a project is deleted, the ID can't be used again

Note: If you use a Gmail account, you can leave the default location set to No organization. If you use a Google Workspace account, choose a location that makes sense for your organization.

2. Next, you'll need to enable billing in the Cloud Console to use Cloud resources/APIs. Running through this codelab won't cost much, if anything at all. To shut down resources to avoid incurring billing beyond this tutorial, you can delete the resources you created or delete the project. New Google Cloud users are eligible for the $300 USD Free Trial program.

Start Cloud Shell

While Google Cloud can be operated remotely from your laptop, in this codelab you will be using Google Cloud Shell, a command line environment running in the Cloud.

From the Google Cloud Console, click the Cloud Shell icon on the top right toolbar:

It should only take a few moments to provision and connect to the environment. When it is finished, you should see something like this:

This virtual machine is loaded with all the development tools you'll need. It offers a persistent 5GB home directory, and runs on Google Cloud, greatly enhancing network performance and authentication. All of your work in this codelab can be done within a browser. You do not need to install anything.

3. Task 1. Setup environment with terraform

We will create a custom VPC with Firewall rules and subnet. Open the cloud console and select the project you will be using.

1. Open Cloud Shell located at the top of your console on the right, ensure you see the correct project id in Cloud Shell, confirm any prompts to allow access.
2. Create a folder called terraform-build and move to folder

mkdir terraform-build  && cd terraform-build

3. Create a main.tf and variable.tf file.

touch main.tf variable.tf 

4. Switch over to the Cloud Shell editor view. Select editor, ensure you allow any necessary prompts so the interface can load.
5. Once loaded navigate to, File > Open Folder and go to the /home/your-user-name/terraform-build and select Ok to open the folder in the editor.
6. Select the variable.tf file and add the following. Replace the your-project-id-here text with your actual project ID in quotes

variable "project_id" {
  type = string
  default = "your-project-id-here"
}

variable "network_id" {
  type = string
  default = "gemini-vpc-net"
}

7. Next open the main.tf file. We are going to add some terraform code to perform various actions as explained below.

Enable API's	resource "google_project_service" "default"
Create VPC called python-net	resource "google_compute_network" "default"
Add a subnet	resource "google_compute_subnetwork" "default"
Add two Firewall rules	resource "google_compute_firewall" "allow_icmp"resource "google_compute_firewall" "allow_ssh"

8. Copy and paste the following into the main .tf file.

resource "google_project_service" "default" {
  for_each = toset([
    "dns.googleapis.com",
    "aiplatform.googleapis.com",
    "servicedirectory.googleapis.com"
  ])

  service            = each.value
  disable_on_destroy = false
}

resource "google_compute_network" "default" {
  project                 = var.project_id
  name                    = var.network_id
  auto_create_subnetworks = false
  mtu                     = 1460
  routing_mode            = "GLOBAL"
}

resource "google_compute_subnetwork" "default" {
  name          = "vm1-subnet"
  ip_cidr_range = "192.168.100.0/24"
  region        = "us-east1"
  stack_type    = "IPV4_ONLY"
  network       = google_compute_network.default.id
}

resource "google_compute_firewall" "allow_icmp" {
  name    = "allow-icmp-${google_compute_network.default.name}"
  network = google_compute_network.default.id
  project = var.project_id

  allow {
    protocol = "icmp"
  }

  source_ranges = ["0.0.0.0/0"]
  target_tags   = ["allow-icmp"]
}

resource "google_compute_firewall" "allow_ssh" {
  name    = "allow-ssh-${google_compute_network.default.name}"
  network = google_compute_network.default.id
  project = var.project_id

  allow {
    protocol = "tcp"
    ports    = ["22"]
  }

  source_ranges = ["0.0.0.0/0"]
  target_tags   = ["allow-ssh"]
}

9. Switch back to Cloud Shell terminal make sure your are in the terraform-build directory cd terraform-build and run the following commands

terraform init

Initializes the working directory. This step downloads the providers required for the given configuration.

terraform plan

Generates an execution plan, showing what actions Terraform will take to deploy your infrastructure.

10. Now to create the resources run the terraform apply command and type yes to run.

Verify your setup: Check your environment for

• VPC called gemini-vpc-net
• subnet
• firewall-rules

4. Task 2. Create NAT gateway and VMs with Terraform

We need to grant outbound external access to the internet so lets create a Cloud NAT gateway and attach it.

1. Open Cloud Shell, navigate to terraform-build folder and create the following files (total three files). We will edit these later on.

touch nat-vm.tf psc.tf dns.tf

2. Switch to Cloud Shell editor view and select the nat-vm.tf file and add the following Terraform code. This will create a NAT gateway and two VMs.

Terraform nat-vm.tf

resource "google_compute_router" "default" {
  name    = "outbound-nat"
  region  = "us-east1"
  network = google_compute_network.default.id

 bgp {
  asn = 64514
  }
}

resource "google_compute_router_nat" "default" {
  name                               = "outbound-gw"
  router                             = google_compute_router.default.name
  region                             = google_compute_router.default.region
  nat_ip_allocate_option             = "AUTO_ONLY"
  source_subnetwork_ip_ranges_to_nat = "ALL_SUBNETWORKS_ALL_IP_RANGES"

  log_config {
    enable = true
    filter = "ERRORS_ONLY"
  }
}

resource "google_compute_instance" "vm1" {
  name         = "cli-vm"
  zone         = "us-east1-b"
  machine_type = "n2-standard-2"

  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-12"
    }
  }

  network_interface {
    subnetwork = google_compute_subnetwork.default.id  
    stack_type = "IPV4_ONLY"
  }

  tags = ["allow-ssh", "allow-icmp"]

  metadata_startup_script = <<-EOF
    sudo apt-get update    
  EOF
}

resource "google_compute_instance" "vm2" {
  name         = "monitor-vm"
  zone         = "us-east1-b"
  machine_type = "n2-standard-2"

  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-12"
    }
  }

  network_interface {
    subnetwork = google_compute_subnetwork.default.id   
    stack_type = "IPV4_ONLY"
  }

  tags = ["allow-ssh", "allow-icmp"]

  metadata_startup_script = <<-EOF
    sudo apt-get update
    sudo apt-get install python3 python3-dev python3-venv -y
    sudo apt-get install tcpdump dnsutils -y
  EOF
}

3. Switch to Cloud Shell terminal, ensure your are in the terraform-build folder and run terraform plan this will show you that 4 items will be added, then run terraform apply and type yes to create the NAT gateway and two vm's.

Verify your setup: Check your environment for

1 NAT gateway

1 cloud router

VM named cli-vm

VM named monitor-vm

5. Task 3. Configure CLI VMs and test

1. Navigate to VM instances. Select the vm starting with cli-vm. Choose SSH.
2. Once you SSH you should have access to the vm let's create a folder for when we run the Gemini CLI

mkdir geminicli && cd geminicli

3. To install Gemini CLI we need Node js. Install node JS using the following

curl -fsSL https://deb.nodesource.com/setup_24.x -o nodesource_setup.sh

Run script

sudo -E bash nodesource_setup.sh

Install Node js

sudo apt-get install -y nodejs

4. Let's set some variables which will help with using the Vertex AI auth option to authenticate this to do some testing later on. p.s Replace YOUR_PROJECT_ID with your actual project ID

cat <<EOF >> ~/.bashrc 

export GOOGLE_CLOUD_PROJECT="YOUR_PROJECT_ID"
export GOOGLE_CLOUD_LOCATION="us-east1" 
export GOOGLE_GENAI_USE_VERTEXAI=true
EOF

Reload bash

source ~/.bashrc

5. Now let's authenticate. Run the following command in the VM, press y when prompted.

gcloud auth application-default login

6. Next copy the url which appears in the starting with https:// open a new tab in your lab browser window and paste the url. Accept the prompts.
7. When you see the following select copy, switch back to the vm cli-vm session and for Enter authorization code: paste the code you copied and press enter to authenticate.

8. Now lets install Gemini CLI run the command

sudo npm install -g @google/gemini-cli

Once complete type gemini to launch the interface, select a theme your prefer and under the Select Auth method choose Vertex AI

9. Once authenticated you can start using Gemini Cli

Note: You can modify the prompt question to something else.

Monitor traffic route to Gemini via monitor vm

1. Navigate to VM instances. Select the vm starting with monitor-vm. Choose SSH.
2. Once you SSH into monitor-vm you should have access
3. Let's use the dig command to test the connection path to Gemini API. We'll use the us-east1-aiplatform.googleapis.com

dig us-east1-aiplatform.googleapis.com

You should see something similar (the address will differ). Note the path is via public IP addresses since the API is a public API.

DON'T COPY

; <<>> DiG 9.18.33-1~deb12u2-Debian <<>> us-east1-aiplatform.googleapis.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58905
;; flags: qr rd ra; QUERY: 1, ANSWER: 14, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;us-east1-aiplatform.googleapis.com. IN A

;; ANSWER SECTION:
us-east1-aiplatform.googleapis.com. 300 IN A    173.194.217.95
us-east1-aiplatform.googleapis.com. 300 IN A    172.217.204.95
us-east1-aiplatform.googleapis.com. 300 IN A    172.217.203.95
us-east1-aiplatform.googleapis.com. 300 IN A    142.250.98.95
us-east1-aiplatform.googleapis.com. 300 IN A    142.251.107.95
us-east1-aiplatform.googleapis.com. 300 IN A    74.125.196.95
us-east1-aiplatform.googleapis.com. 300 IN A    173.194.216.95
us-east1-aiplatform.googleapis.com. 300 IN A    108.177.11.95
us-east1-aiplatform.googleapis.com. 300 IN A    192.178.219.95
us-east1-aiplatform.googleapis.com. 300 IN A    173.194.210.95
us-east1-aiplatform.googleapis.com. 300 IN A    173.194.212.95
us-east1-aiplatform.googleapis.com. 300 IN A    173.194.215.95
us-east1-aiplatform.googleapis.com. 300 IN A    108.177.12.95
us-east1-aiplatform.googleapis.com. 300 IN A    74.125.26.95

4. Now let's do a quick ping test to see if we can connect to the Gemini API. This command will use 4 pings to the us-east1-aiplatform.googleapis.com so we will get a response from the public address of the API.

ping -c 4 us-east1-aiplatform.googleapis.com

5. We will come back to test this VM later on. Close off the SSH session and let's continue.

6. Task 4. Create the PSC endpoint to googleapis with Terraform

To enable private connectivity to our Vertex API endpoint we will create a Private Service Connect endpoint for Google APIs. This will allow us to use a private IP address we assign to route traffic to the Google APIs we need, in this case Vertex.

1. Open Cloud Shell in editor view if not open already. We are going to create the following:

• Create an IP for the PSC endpoint 10.10.100.250 (resource "google_compute_global_address" "default")
• Create a PSC endpoint to Google APIs (resource "google_compute_global_forwarding_rule" "default")

Open the psc.tf file in the terraform-build folder. Add the following code to file.

Terraform psc.tf

resource "google_compute_global_address" "default" {
  name         = "gemini-ip"
  purpose      = "PRIVATE_SERVICE_CONNECT"
  network      = google_compute_network.default.id
  address_type = "INTERNAL"
  address      = "10.10.100.250"
}

resource "google_compute_global_forwarding_rule" "default" {  
  name                  = "pscgemini"
  target                = "all-apis"
  network               = google_compute_network.default.id
  ip_address            = google_compute_global_address.default.id
  load_balancing_scheme = ""
  }

2. Switch to Cloud Shell terminal, ensure your are in the terraform-build folder. Then run terraform init Then run terraform plan this will show you that 2 items will be added,
   then run terraform apply and type yes to create the IP and PSC Google APIs endpoint.
3. Verify endpoint exist

gcloud compute addresses list --filter="name=( 'gemini-ip' ...)"

gcloud compute forwarding-rules describe pscgemini --global

7. Task 5. Create Manual DNS entry to googleapis with Terraform

You can create a manual DNS entry to point to the PSC endpoint using private DNS. This would affect all the networks you assign to it.

1. Navigate to Network services and select Cloud DNS.
2. In zones you should see an automatically created zone for the Private Service Connect for Google APIs, with zone type service directory. This can be used to connect to the PSC endpoint with the format **SERVICE-ENDPOINT.p.googleapis.com Example: aiplatform-pscgemini.p.googleapis.com
3. In this case we want to manually create a private DNS entry. The config will be as follows

• Create a private DNS zone named "googleapis-private" for "googleapis.com" and restrict it to the "python-net" network.
• Add an A record to map "googleapis.com" to the IP address "10.10.100.250".
• Add a CNAME record to redirect all subdomains of "googleapis.com" (e.g., www.googleapis.com) to "googleapis.com".

4. Open Cloud Shell in editor view if not open already. Open the dns.tf file in the terraform-build folder. Add the following code to file.

Terraform dns.tf

resource "google_dns_managed_zone" "private_zone" {
  name        = "googleapis-private"
  dns_name    = "googleapis.com."  
  visibility  = "private"
  project     = var.project_id     

  private_visibility_config {
    networks {
      network_url = google_compute_network.default.id  
    }
  }
}

resource "google_dns_record_set" "a_record" {
  name    = "googleapis.com."  
  type    = "A"
  ttl     = 300
  managed_zone = google_dns_managed_zone.private_zone.name
  project = var.project_id    

  rrdatas = ["10.10.100.250"]
}

resource "google_dns_record_set" "cname_record" {
 name    = "*.googleapis.com."
 type    = "CNAME"
 ttl     = 300
 managed_zone = google_dns_managed_zone.private_zone.name
 project = var.project_id    

 rrdatas = ["googleapis.com."]  
}

5. Switch to Cloud Shell terminal, ensure your are in the terraform-build folder. Then run terraform plan this will show you what items will be added,
   Then run terraform apply and type yes to create the private DNS entry.
6. You should see a set up with an A record and CNAME like this see

7. Next we verify connectivity with these changes on monitor-vm

8. Task 7. Verify endpoint connectivity via IP address

Let's connect using the private endpoint to connect to Gemini.

Note: No manual configurations are necessary on the VM.

1. Go to VM Instance monitor-vm. Select SSH and SSH into the VM
2. Check the connectivity path to us-east1-aiplatform.googleapis.com using the ping command. This will ping the IP address in the private DNS, A record for googleapis. This IP a PSC endpoint and your pings will be unsuccessful.

ping -c 2 us-east1-aiplatform.googleapis.com

3. Check the connectivity path with a ping using the automatically created DNS entry for PSC Google APIs with aiplatform-pscvertexgemini.p.googleapis.com. This points to the IP address of the PSC endpoint and your pings will be unsuccessful.

ping -c 2 aiplatform-pscgemini.p.googleapis.com

4. Check the connectivity path to us-east1-aiplatform.googleapis.com using the dig command. This should be the IP address of the PSC endpoint (10.10.100.250).

dig us-east1-aiplatform.googleapis.com

5. Once completed you can close the monitor-vm SSH session.
6. Now we can run a TCP dump on the cli-vm. We would need to open two SSH sessions to the same vm. One session will run the tcpdump command the other we will use Gemini CLI.
7. SSH into the first session on the cli-vm
8. Type the follow command to see the connectivity in a tcpdump

sudo tcpdump -i any port 53 -n or host us-east1-aiplatform.googleapis.com

9. Now SSH into next session on the the cli-vm
10. Activate the Gemini CLI by typing gemini
11. Ask the following question, what color is the sky to generate a call to the Gemini API

12. Press enter to run and see the result.
13. Switch back over to the first session on the cli-vm. You should see the result of the tcpdump. You will notice in and out and the IP address of the VM is using the PSC endpoint IP address to connect to gemini API

Close all SSH sessions to VM Instances

9. Clean Up

1. Go to Cloud Shell make sure you are in the terraform-build directory cd terraform-build and run the following command terraform destroy and type yes all the resources you created in your project with Terraform will be removed.

10. Congratulations

Congratulations, you've successfully connected to Gemini CLI using both public API address and privately using Private Service Connect Endpoint for Google APIs. This functionality can extend private API connectivity into your on-prem/other cloud environment which are connected via (Interconnect, Cross-Cloud Interconnect and VPC).

Next steps / Learn more

You can read more about Vertex AI networking and check out the Gemini CLI repo

Take your next lab

Continue your quest with Google Cloud, and check out these other Google Cloud labs:

• Networking Fundamentals in Google Cloud
• Generative AI Explorer - Vertex AI
• Tour of Cloud Networking