Access Gemini 3 Pro chat using python sdk via Private Service Connect endpoint

1. Overview

Vertex AI API can be accessed over the internet, however, in your enterprise you may want to access the Vertex AI API's privately without going over the internet. In this lab you will first.

• Access the Gemini 3 Pro API via Vertex using python sdk
• This will be running on a VM instance
• Connection will be via Cloud NAT to the public internet.

Then you will create a Private Service Connect endpoint to Google APIs, and change the traffic flow to use the private endpoint to connect to the Gemini chat API. The configurations will be a combination of Terraform, gcloud and console.

In this lab, you're going to be creating the following pattern.

Figure1.

2. Objective

In this lab you will learn how to perform the following task:

• Set up VM instance to use python sdk
• Connect to Gemini chat via python script
• Configure PSC endpoint to connect to Googleapis
• Verify connectivity path to Googleais
• Configure manual DNS entries

Self-paced environment setup

1. Sign-in to the Google Cloud Console and create a new project or reuse an existing one. If you don't already have a Gmail or Google Workspace account, you must create one.

• The Project name is the display name for this project's participants. It is a character string not used by Google APIs. You can always update it.
• The Project ID is unique across all Google Cloud projects and is immutable (cannot be changed after it has been set). The Cloud Console auto-generates a unique string; usually you don't care what it is. In most codelabs, you'll need to reference your Project ID (typically identified as PROJECT_ID). If you don't like the generated ID, you might generate another random one. Alternatively, you can try your own, and see if it's available. It can't be changed after this step and remains for the duration of the project.
• For your information, there is a third value, a Project Number, which some APIs use. Learn more about all three of these values in the documentation.

Caution: A project ID is globally unique and can't be used by anyone else after you've selected it. You are the only user of that ID. Even if a project is deleted, the ID can't be used again

Note: If you use a Gmail account, you can leave the default location set to No organization. If you use a Google Workspace account, choose a location that makes sense for your organization.

2. Next, you'll need to enable billing in the Cloud Console to use Cloud resources/APIs. Running through this codelab won't cost much, if anything at all. To shut down resources to avoid incurring billing beyond this tutorial, you can delete the resources you created or delete the project. New Google Cloud users are eligible for the $300 USD Free Trial program.

Start Cloud Shell

While Google Cloud can be operated remotely from your laptop, in this codelab you will be using Google Cloud Shell, a command line environment running in the Cloud.

From the Google Cloud Console, click the Cloud Shell icon on the top right toolbar:

It should only take a few moments to provision and connect to the environment. When it is finished, you should see something like this:

This virtual machine is loaded with all the development tools you'll need. It offers a persistent 5GB home directory, and runs on Google Cloud, greatly enhancing network performance and authentication. All of your work in this codelab can be done within a browser. You do not need to install anything.

3. Task 1. Setup Environment with terraform

We will create a custom VPC with Firewall rules and subnet. Open the cloud console and select the project you will be using.

1. Open Cloud Shell located at the top of your console on the right, ensure you see the correct project id in Cloud Shell, confirm any prompts to allow access.
2. Create a folder called terraform-build and move to folder

mkdir terraform-build  && cd terraform-build

3. Create a main.tf and variable.tf file.

touch main.tf variable.tf 

4. Switch over to the Cloud Shell editor view. Select editor, ensure you allow any necessary prompts so the interface can load.
5. Once loaded navigate to, File > Open Folder and go to the /home/your-user-name/terraform-build and select Ok to open the folder in the editor.
6. Select the variable.tf file and add the following. Replace the your-project-id-here text with your actual project ID in quotes

variable "project_id" {
  type = string
  default = "your-project-id-here"
}

variable "network_id" {
  type = string
  default = "python-net"
}

7. Next open the main.tf file. We are going to add some terraform code to perform various actions as explained below.

Enable API's	resource "google_project_service" "default"
Create VPC called python-net	resource "google_compute_network" "default"
Add a subnet	resource "google_compute_subnetwork" "default"
Add two Firewall rules	resource "google_compute_firewall" "allow_icmp"resource "google_compute_firewall" "allow_ssh"

8. Copy and paste the following into the main .tf file.

resource "google_project_service" "default" {
  project = var.project_id 
  for_each = toset([
    "dns.googleapis.com",
    "aiplatform.googleapis.com",
    "servicedirectory.googleapis.com"
  ])

  service            = each.value
  disable_on_destroy = false
}

resource "google_compute_network" "default" {
  project                 = var.project_id
  name                    = var.network_id
  auto_create_subnetworks = false
  mtu                     = 1460
  routing_mode            = "GLOBAL"
}

resource "google_compute_subnetwork" "default" {
  name          = "vm1-subnet"
  ip_cidr_range = "10.0.11.0/24"
  project       = var.project_id
  region        = "us-east1"
  stack_type    = "IPV4_ONLY"
  network       = google_compute_network.default.id
}

resource "google_compute_firewall" "allow_icmp" {
  name    = "allow-icmp-${google_compute_network.default.name}"
  network = google_compute_network.default.id
  project = var.project_id

  allow {
    protocol = "icmp"
  }

  source_ranges = ["0.0.0.0/0"]
  target_tags   = ["allow-icmp"]
}

resource "google_compute_firewall" "allow_ssh" {
  name    = "allow-ssh-${google_compute_network.default.name}"
  network = google_compute_network.default.id
  project = var.project_id

  allow {
    protocol = "tcp"
    ports    = ["22"]
  }

  source_ranges = ["0.0.0.0/0"]
  target_tags   = ["allow-ssh"]
}

9. Switch back to Cloud Shell terminal make sure your are in the terraform-build directory cd terraform-build and run the following commands

terraform init

Initializes the working directory. This step downloads the providers required for the given configuration.

terraform plan

Generates an execution plan, showing what actions Terraform will take to deploy your infrastructure.

10. Now to create the resources run the terraform apply command and type yes to run.

Verify your setup: Check your environment for

1 VPC called python-net

1 subnet

2 firewall-rules

4. Task 2. Create NAT gateway and VMs with Terraform

We need to grant outbound external access to the internet so lets create a Cloud NAT gateway and attach it.

1. Open Cloud Shell, navigate to terraform-build folder and create the following files (total three files). We will edit these later on.

touch nat-vm.tf psc.tf dns.tf

2. Switch to Cloud Shell editor view and select the nat-vm.tf file and add the following Terraform code. This will create a NAT gateway and two VMs.

Terraform nat-vm.tf

resource "google_compute_router" "default" {
  name    = "py-outbound-nat"
  region  = "us-east1"
  network = google_compute_network.default.id
  project = var.project_id
  

 bgp {
  asn = 64514
  }
}

resource "google_compute_router_nat" "default" {
  name                               = "py-outbound-nat-gw"
  router                             = google_compute_router.default.name
  region                             = google_compute_router.default.region
  nat_ip_allocate_option             = "AUTO_ONLY"
  project                            = var.project_id
  source_subnetwork_ip_ranges_to_nat = "ALL_SUBNETWORKS_ALL_IP_RANGES"

  log_config {
    enable = true
    filter = "ERRORS_ONLY"
  }
}

resource "google_compute_instance" "vm1" {
  name         = "py-vm1"
  zone         = "us-east1-b"
  machine_type = "n2-standard-2"
  project      = var.project_id

  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-12"
    }
  }

  network_interface {
    subnetwork = google_compute_subnetwork.default.id  
    stack_type = "IPV4_ONLY"
  }

  tags = ["allow-ssh", "allow-icmp"]

  metadata_startup_script = <<-EOF
    #! /bin/bash
    sudo apt-get update
    sudo apt-get install python3 python3-dev python3-venv git -y
    sudo apt-get install tcpdump dnsutils -y    
    
      mkdir -p ~/py-gem-env
      cd ~/py-gem-env
      python3 -m venv env
      source env/bin/activate
      pip install --upgrade pip
      pip install ipython google-genai
    '
  EOF
}

3. Switch to Cloud Shell terminal, ensure your are in the terraform-build folder and run terraform plan this will show you that 4 items will be added, then run terraform apply and type yes to create the NAT gateway and vm.

Verify your setup: Check your environment for

1 NAT gateway

1 cloud router

VM named py-vm1

5. Task 3. Configure VMs and test

1. Navigate to VM instances. Select the vm starting with py-vm1. Choose SSH.
2. Once you SSH into py-vm1, enable root by typing sudo -i
3. Activate your venv environment:

cd py-gem-env
source env/bin/activate

4. Now let's authenticate this to do some testing later on. Run the following command in the VM, press y when prompted.

gcloud auth application-default login

5. Next copy the url which appears in the starting with https:// open a new tab in your lab browser window and paste the url. Accept the prompts.
6. When you see the following select copy, switch back to the vm py-vm1 session and for Enter authorization code: paste the code you copied and press enter to authenticate.

7. Now let's do a quick test to see if we can connect to the Vertex API, this uses the *-aiplatform.googleapis.com so we will do a dig to that address to see how the traffic routes.

dig *-aiplatform.googleapis.com

8. You should see something similar (the address will differ). Note the path is via public IP addresses since the API is a public API. DON'T COPY

; <<>> DiG 9.18.41-1~deb12u1-Debian <<>> *-aiplatform.googleapis.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 60947
;; flags: qr rd ra; QUERY: 1, ANSWER: 16, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;*-aiplatform.googleapis.com.   IN      A

;; ANSWER SECTION:
*-aiplatform.googleapis.com. 300 IN     A       173.194.216.95
*-aiplatform.googleapis.com. 300 IN     A       108.177.11.95
*-aiplatform.googleapis.com. 300 IN     A       192.178.219.95
*-aiplatform.googleapis.com. 300 IN     A       74.125.134.95
*-aiplatform.googleapis.com. 300 IN     A       74.125.139.95
*-aiplatform.googleapis.com. 300 IN     A       108.177.12.95
*-aiplatform.googleapis.com. 300 IN     A       173.194.210.95
*-aiplatform.googleapis.com. 300 IN     A       74.125.26.95
*-aiplatform.googleapis.com. 300 IN     A       173.194.212.95
*-aiplatform.googleapis.com. 300 IN     A       172.217.204.95

9. Now let's use python. Type ipython to activate the ipython interface.

ipython

10. Now copy and paste the following. This asks Gemini "What are all the colors of the Google logo" and "What is the color of the sky" .

from google import genai
from google.genai import types
import os
import sys

LOCATION = "global"
MODEL_ID = "gemini-3-pro-preview" 

try:
    client = genai.Client(vertexai=True, location=LOCATION)
    print(f"Successfully initialized Google Gen AI Client (Vertex AI mode) in {LOCATION}")
except Exception as e:
    print(f"Error initializing client: {e}")
    print("Ensure you have installed the library: `pip install google-genai`")
    print("And authenticated: `gcloud auth application-default login`")
    sys.exit(1)

class SimpleChatSession:
    def __init__(self, model_id):
        self.model_id = model_id
        self.history = [] 

    def send_message(self, prompt, stream=True):
        user_content = types.Content(
            role="user",
            parts=[types.Part(text=prompt)]
        )
        self.history.append(user_content)

        try:
            response_stream = client.models.generate_content_stream(
                model=self.model_id,
                contents=self.history,
                config=types.GenerateContentConfig(
                    temperature=0.7 
                )
            )

            accumulated_text = ""
            
            for chunk in response_stream:
                if chunk.text:
                    yield chunk.text
                    accumulated_text += chunk.text
            
            model_content = types.Content(
                role="model",
                parts=[types.Part(text=accumulated_text)]
            )
            self.history.append(model_content)

        except Exception as e:
            print(f"\n[Error during generation: {e}]")

def get_chat_response(session: SimpleChatSession, prompt: str):
    print(f"\n--- User: {prompt} ---")
    print("--- Model: ", end="")
    
    stream_generator = session.send_message(prompt)
    
    full_text = ""
    for chunk_text in stream_generator:
        full_text += chunk_text
        print(chunk_text, end="", flush=True)

    print() 
    return full_text

if __name__ == "__main__":
    chat_session = SimpleChatSession(MODEL_ID)

    get_chat_response(chat_session, "Hello.")
    get_chat_response(chat_session, "What are all the colors of the Google logo?")
    get_chat_response(chat_session, "What color is the sky?")

11. Press enter twice to run and see the result.
12. This request accessed Vertex via the public API.
13. Close off SSH session let's continue.

Note: You can modify the prompt question to something else.

6. Task 4. Create the PSC endpoint to googleapis with Terraform

To enable private connectivity to our Vertex API endpoint we will create a Private Service Connect endpoint for Google APIs. This will allow us to use a private IP address we assign to route traffic to the Google APIs we need, in this case Vertex.

1. Open Cloud Shell in editor view if not open already. We are going to create the following:

• Create an IP for the PSC endpoint 192.168.255.250 (resource "google_compute_global_address" "default")
• Create a PSC endpoint to Google APIs (resource "google_compute_global_forwarding_rule" "default")

Open the psc.tf file in the terraform-build folder. Add the following code to file.

Terraform psc.tf

resource "google_compute_global_address" "default" {
  name         = "vertex-ip"
  purpose      = "PRIVATE_SERVICE_CONNECT"
  network      = google_compute_network.default.id
  address_type = "INTERNAL"
  address      = "192.168.255.250"
}

resource "google_compute_global_forwarding_rule" "default" {  
  name                  = "pscvertexgemini"
  target                = "all-apis"
  network               = google_compute_network.default.id
  ip_address            = google_compute_global_address.default.id
  load_balancing_scheme = ""
  }

2. Switch to Cloud Shell terminal, ensure your are in the terraform-build folder. Then run terraform init Then run terraform plan this will show you that 2 items will be added,
   then run terraform apply and type yes to create the IP and PSC Google APIs endpoint.
3. Verify endpoint exist

gcloud compute addresses list --filter="name=( 'vertex-ip' ...)"

gcloud compute forwarding-rules describe pscvertexgemini --global

7. Task 5. Create Manual DNS entry to googleapis with Terraform

You can create a manual DNS entry to point to the PSC endpoint using private DNS. This would affect all the networks you assign to it.

1. Navigate to Network services and select Cloud DNS.
2. In zones you should see an automatically created zone for the Private Service Connect for Google APIs, with zone type service directory. This can be used to connect to the PSC endpoint with the format **SERVICE-ENDPOINT.p.googleapis.com Example: aiplatform-pscvertexgemini.p.googleapis.com
3. In this case we want to manually create a private DNS entry. The config will be as follows

• Create a private DNS zone named "googleapis-private" for "googleapis.com" and restrict it to the "python-net" network.
• Add an A record to map "googleapis.com" to the IP address "192.168.255.250".
• Add a CNAME record to redirect all subdomains of "googleapis.com" (e.g., www.googleapis.com) to "googleapis.com".

4. Open Cloud Shell in editor view if not open already. Open the dns.tf file in the terraform-build folder. Add the following code to file.

Terraform dns.tf

resource "google_dns_managed_zone" "private_zone" {
  name        = "googleapis-private"
  dns_name    = "googleapis.com."  
  visibility  = "private"
  project     = var.project_id     

  private_visibility_config {
    networks {
      network_url = google_compute_network.default.id  
    }
  }
}

resource "google_dns_record_set" "a_record" {
  name    = "googleapis.com."  
  type    = "A"
  ttl     = 300
  managed_zone = google_dns_managed_zone.private_zone.name
  project = var.project_id    

  rrdatas = ["192.168.255.250"]
}

resource "google_dns_record_set" "cname_record" {
 name    = "*.googleapis.com."
 type    = "CNAME"
 ttl     = 300
 managed_zone = google_dns_managed_zone.private_zone.name
 project = var.project_id    

 rrdatas = ["googleapis.com."]  
}

5. Switch to Cloud Shell terminal, ensure your are in the terraform-build folder. Then run terraform plan this will show you what items will be added,
   Then run terraform apply and type yes to create the private DNS entry.
6. You should see a set up with an A record and CNAME like this see
7. Next we verify connectivity with these changes on py-vm1

8. Task 6. Verify endpoint connectivity via IP address

Let's connect using the private endpoint to connect to Gemini.

Note: No manual configurations are necessary on the VM.

1. Go to VM Instance py-vm1. Select SSH and SSH into the VM
2. Gain root access by typing sudo -i
3. Check the connectivity path to aiplatform.googleapis.com using the ping command. This will ping the IP address in the private DNS, A record for googleapis. This IP a PSC endpoint and your pings will be unsuccessful.

ping -c 2 aiplatform.googleapis.com

4. Check the connectivity path with a ping using the automatically created DNS entry for PSC Google APIs with aiplatform-pscvertexgemini.p.googleapis.com. This points to the IP address of the PSC endpoint and your pings will be unsuccessful.

ping -c 2 aiplatform-pscvertexgemini.p.googleapis.com

5. Check the connectivity path to aiplatform.googleapis.com using the dig command. This should be the IP address of the PSC endpoint.

dig aiplatform.googleapis.com

6. Go back to the console and open another instance of VM Instance py-vm1. Select SSH and SSH into the VM
7. Gain root access by typing sudo -i
8. Run the follow command to see the connectivity in a TCP dump

sudo tcpdump -i any port 53 -n or host aiplatform.googleapis.com

9. Now switch back to the first SSH instance of VM Instance py-vm1
10. Activate the env using

cd py-gem-env
source env/bin/activate

11. Now let's test python. Type ipython to activate the ipython interface.

ipython

12. Now copy and paste the following. This asks Gemini "In one short sentence summarize what is a tokenizer in the context of AI?" and "Are whales awesome or not?".

from google import genai
from google.genai import types
import os
import sys

LOCATION = "global"
MODEL_ID = "gemini-3-pro-preview" 

try:
    client = genai.Client(vertexai=True, location=LOCATION)
    print(f"Successfully initialized Google Gen AI Client (Vertex AI mode) in {LOCATION}")
except Exception as e:
    print(f"Error initializing client: {e}")
    print("Ensure you have installed the library: `pip install google-genai`")
    print("And authenticated: `gcloud auth application-default login`")
    sys.exit(1)

class SimpleChatSession:
    def __init__(self, model_id):
        self.model_id = model_id
        self.history = [] 

    def send_message(self, prompt, stream=True):
        user_content = types.Content(
            role="user",
            parts=[types.Part(text=prompt)]
        )
        self.history.append(user_content)

        try:
            response_stream = client.models.generate_content_stream(
                model=self.model_id,
                contents=self.history,
                config=types.GenerateContentConfig(
                    temperature=0.7 
                )
            )

            accumulated_text = ""
            
            for chunk in response_stream:
                if chunk.text:
                    yield chunk.text
                    accumulated_text += chunk.text
            
            model_content = types.Content(
                role="model",
                parts=[types.Part(text=accumulated_text)]
            )
            self.history.append(model_content)

        except Exception as e:
            print(f"\n[Error during generation: {e}]")

def get_chat_response(session: SimpleChatSession, prompt: str):
    print(f"\n--- User: {prompt} ---")
    print("--- Model: ", end="")
    
    stream_generator = session.send_message(prompt)
    
    full_text = ""
    for chunk_text in stream_generator:
        full_text += chunk_text
        print(chunk_text, end="", flush=True)

    print() 
    return full_text

if __name__ == "__main__":
    chat_session = SimpleChatSession(MODEL_ID)

    get_chat_response(chat_session, "Hello.")
    get_chat_response(chat_session, "In one short sentence summarize what is a tokenizer in the context of AI?")
    get_chat_response(chat_session, "Are whales awesome or not?")

13. Press enter twice to run and see the result.
14. Switch back over to the second instance of VM Instance py-vm1. You should see the result of the TCPDUMP. You will notice in and out and the IP address of the VM is using the PSC endpoint IP address to connect to aiplatform.googleapis.com

Close all SSH sessions to VM Instance py-vm1

9. Clean Up

1. Go to Cloud Shell make sure you are in the terraform-build directory cd terraform-build
2. Run the terraform plan destroy to see all the changes that will be made

terraform plan -destroy

3. Then run the command terraform destroy and then type yes and all the resources you created in your project with Terraform will be removed.

terraform destroy 

10. Congratulations

Congratulations, you've successfully connected to Vertex and used Gemini 3 pro via public API address and privately using Private Service Connect Endpoint for Google APIs. This functionality can extend private API connectivity into your on-prem/other cloud environment which are connected via (Interconnect, Cross-Cloud Interconnect and VPC).

Next steps / Learn more

You can read more about Vertex AI networking

Codelab: Access Anthropic Claude on Vertex AI with python sdk via Private Service Connect endpoint

Take your next lab

Continue your quest with Google Cloud, and check out these other Google Cloud Skills Boost labs:

• Networking Fundamentals in Google Cloud
• Generative AI Explorer - Vertex AI
• Agentic AI on Google Cloud