1. Introduction
Private Service Connect is a capability of Google Cloud networking that allows consumers to access managed services privately from inside their VPC network. Similarly, it allows managed service producers to host these services in their own separate VPC networks and offer a private connection to their consumers.
Until now, once PSC services are exposed through a service attachment, changes could not be made to service load balancers without removing the service attachment and disrupting the consumer PSC endpoint connectivity. With the introduction of PSC mutability, it enables producers to update their load balancer while preserving the PSC endpoint connection. Maintaining the PSC endpoint connection to the service attachment while changing the load balancer has the benefit of not requiring any action on the consumer side.
At this time, PSC mutability will support
- Migration to a load balancer forwarding rule of the same type (ex. Network Passthrough to Network Passthrough)
- Migration to a load balancer forwarding rule of a different type (ex. Network Passthrough to Application load balancer)
PSC mutability does not support editing the forwarding rule in place.
In this lab, you'll create a producer web service exposed through an internal Network Passthrough load balancer, ensure that the service has connectivity through a PSC endpoint, then update the service attachment association to a new forwarding rule to an internal Application load balancer.
What you'll learn
- Create a simple Apache web service exposed as a PSC producer service.
- Create a PSC endpoint.
- Create a Cloud DNS private zone for the consumer service calls.
- Update the forwarding rule associated with the service attachment using PSC Mutability.
What you'll need
- A Google Cloud project with owner permissions
- Basic knowledge of Google Cloud networking
2. Test Environment
Traditionally, Producers and Consumers work in different projects. For simplicity purposes, we will perform all actions in the same project, but each of the steps will be labeled with Producer or Consumer projects in case the user prefers to work in different projects.
For this lab, we will start by creating producer-vpc with three subnets; one to host the producer service and test VM, one for the load balancer forwarding rules, and a PSC NAT subnet. We'll need a Cloud Router and Cloud NAT for internet reachability to download Linux packages. We'll expose our Apache service, configured through an unmanaged instance group, through an internal regional Network Passthrough load balancer and associate the forwarding rule with a service attachment.
On the consumer side, we'll create consumer-vpc with a single subnet to host our PSC endpoint and a test client VM. We'll configure a Cloud DNS private zone to access the service via a hostname.
To showcase the PSC Mutability feature, on the producer side, we'll add two additional subnets; one to host the new service VM, and one for the load balancer proxy-only subnet. We'll create a new VM hosting an Apache web server, again configured through an unmanaged instance group, and expose it through a new internal regional Application load balancer. We'll update the service attachment to point to the new forwarding rule and test that the connectivity on the consumer side remains in place.

3. Setup and Requirements
Self-paced environment setup
- Sign-in to the Google Cloud Console and create a new project or reuse an existing one. If you don't already have a Gmail or Google Workspace account, you must create one.



- The Project name is the display name for this project's participants. It is a character string not used by Google APIs. You can always update it.
- The Project ID is unique across all Google Cloud projects and is immutable (cannot be changed after it has been set). The Cloud Console auto-generates a unique string; usually you don't care what it is. In most codelabs, you'll need to reference your Project ID (typically identified as
PROJECT_ID). If you don't like the generated ID, you might generate another random one. Alternatively, you can try your own, and see if it's available. It can't be changed after this step and remains for the duration of the project. - For your information, there is a third value, a Project Number, which some APIs use. Learn more about all three of these values in the documentation.
- Next, you'll need to enable billing in the Cloud Console to use Cloud resources/APIs. Running through this codelab won't cost much, if anything at all. To shut down resources to avoid incurring billing beyond this tutorial, you can delete the resources you created or delete the project. New Google Cloud users are eligible for the $300 USD Free Trial program.
Start Cloud Shell
While Google Cloud can be operated remotely from your laptop, in this codelab you will be using Google Cloud Shell, a command line environment running in the Cloud.
From the Google Cloud Console, click the Cloud Shell icon on the top right toolbar:

It should only take a few moments to provision and connect to the environment. When it is finished, you should see something like this:

This virtual machine is loaded with all the development tools you'll need. It offers a persistent 5GB home directory, and runs on Google Cloud, greatly enhancing network performance and authentication. All of your work in this codelab can be done within a browser. You do not need to install anything.
4. Before you begin
Enable APIs
Inside Cloud Shell, make sure that your project id is set up
gcloud config list project gcloud config set project [YOUR-PROJECT-NAME] export project=[YOUR-PROJECT-NAME] export region=us-central1 export zone=$region-a echo $project echo $region echo $zone
Enable all necessary services
gcloud services enable compute.googleapis.com gcloud services enable dns.googleapis.com
5. Build Producer Network [Producer Project]
Create VPC Network
From Cloud Shell
gcloud compute networks create producer-vpc \
--subnet-mode=custom
Create Subnets
We'll start by deploying 3 subnets into the producer-vpc. One will be for deploying the producer service, one for the load balancer forwarding rules, and one that will be associated with the PSC Service Attachment for our PSC NAT subnet.
From Cloud Shell
gcloud compute networks subnets create producer-service-subnet \
--network=producer-vpc \
--range=10.0.0.0/28 \
--region=$region
gcloud compute networks subnets create producer-fr-subnet \
--network=producer-vpc \
--range=192.168.0.0/28 \
--region=$region
gcloud compute networks subnets create psc-nat-subnet \
--network=producer-vpc \
--range=10.100.0.0/28 \
--region=$region \
--purpose=PRIVATE_SERVICE_CONNECT
Create Cloud NAT
A Cloud NAT is required to install the proper packages for our producer services.
From Cloud Shell
gcloud compute routers create $region-cr \
--network=producer-vpc \
--region=$region
From Cloud Shell
gcloud compute routers nats create $region-nat \
--router=$region-cr \
--region=$region \
--nat-all-subnet-ip-ranges \
--auto-allocate-nat-external-ips
Create Network Firewall Policy and Rules
From Cloud Shell
gcloud compute network-firewall-policies create producer-vpc-policy --global
gcloud compute network-firewall-policies associations create \
--firewall-policy producer-vpc-policy \
--network producer-vpc \
--name network-producer-vpc \
--global-firewall-policy
To allow IAP to connect to your VM instances, create a firewall rule that:
- Applies to all VM instances that you want to be accessible by using IAP.
- Allows ingress traffic from the IP range 35.235.240.0/20. This range contains all IP addresses that IAP uses for TCP forwarding.
From Cloud Shell
gcloud compute network-firewall-policies rules create 1000 \
--action ALLOW \
--firewall-policy producer-vpc-policy \
--description "SSH with IAP" \
--direction INGRESS \
--src-ip-ranges 35.235.240.0/20 \
--layer4-configs tcp:22 \
--global-firewall-policy
We will start with three firewall rules. One will allow access to VMs for the load balancer health checks (2000), one to allow access to the VMs from the PSC NAT range (3000), and one to allow connectivity between VMs in the service subnet for testing purposes (4000). We will make all rules for ports 80 as we will be using this port for our service.
From Cloud Shell
gcloud compute network-firewall-policies rules create 2000 \
--action ALLOW \
--firewall-policy producer-vpc-policy \
--description "LB healthchecks" \
--direction INGRESS \
--src-ip-ranges 130.211.0.0/22,35.191.0.0/16 \
--layer4-configs tcp:80 \
--global-firewall-policy
gcloud compute network-firewall-policies rules create 3000 \
--action ALLOW \
--firewall-policy producer-vpc-policy \
--description "allow access from PSC NAT subnet" \
--direction INGRESS \
--src-ip-ranges 10.100.0.0/28 \
--layer4-configs tcp:80 \
--global-firewall-policy
gcloud compute network-firewall-policies rules create 4000 \
--action ALLOW \
--firewall-policy producer-vpc-policy \
--description "allow access between producer service VMs and testing client for testing purposes" \
--direction INGRESS \
--src-ip-ranges 10.0.0.0/28 \
--layer4-configs tcp:80 \
--global-firewall-policy
6. Create Producer Service [Producer Project]
We'll create a simple Apache web service using an unmanaged instance group that displays "I am a Producer Service."
Create Instance
From Cloud Shell
gcloud compute instances create producer-service-vm \
--network producer-vpc \
--subnet producer-service-subnet \
--zone $zone \
--no-address \
--metadata startup-script='#! /bin/bash
sudo apt-get update
sudo apt-get install apache2 -y
a2enmod ssl
sudo a2ensite default-ssl
echo "I am a Producer Service." | \
tee /var/www/html/index.html
systemctl restart apache2'
Create Unmanaged Instance Group
From Cloud Shell
gcloud compute instance-groups unmanaged create producer-uig \ --zone=$zone
Add producer-service-vm to the unmanaged instance group we just created.
From Cloud Shell
gcloud compute instance-groups unmanaged add-instances producer-uig \ --zone=$zone \ --instances=producer-service-vm
Create Load Balancer Components
For version 1 of the producer service, we'll expose the service using a Network Passthrough Load Balancer on port 80.
Deploy the health check for port 80.
From Cloud Shell
gcloud compute health-checks create http producer-hc \
--region=$region
Deploy the backend service and add the backend to the backend service.
From Cloud Shell
gcloud compute backend-services create producer-bes \ --load-balancing-scheme=internal \ --protocol=tcp \ --region=$region \ --health-checks=producer-hc \ --health-checks-region=$region gcloud compute backend-services add-backend producer-bes \ --region=$region \ --instance-group=producer-uig \ --instance-group-zone=$zone
Create the static IP address that will be used for the load balancer forwarding rule.
From Cloud Shell
gcloud compute addresses create producer-fr-ip\ --region $region \ --subnet producer-fr-subnet \ --addresses 192.168.0.2
Create the load balancer forwarding rule exposing port 80, using the backend service and IP address we previously created.
From Cloud Shell
gcloud compute forwarding-rules create producer-fr \ --region=$region \ --load-balancing-scheme=internal \ --network=producer-vpc \ --subnet=producer-fr-subnet \ --address=producer-fr-ip \ --ip-protocol=TCP \ --ports=80 \ --backend-service=producer-bes \ --backend-service-region=$region
7. Test the Producer Service [Producer Project]
Create a test VM in the producer-service-subnet, log into the VM and test a call to the Apache service.
From Cloud Shell
gcloud compute instances create producer-test-vm \
--zone=$zone \
--subnet=producer-service-subnet \
--no-address
From Cloud Shell
gcloud compute ssh producer-test-vm \
--zone=$zone \
--command="curl -s 192.168.0.2/index.html"
Expected output
I am a Producer Service.
8. Expose Producer Service Through Service Attachment [Producer Project]
Create the service attachment. Associate it with the forwarding rule we created using the psc-nat-subnet.
From Cloud Shell
gcloud compute service-attachments create producer-attachment \
--region=$region \
--producer-forwarding-rule=producer-fr \
--connection-preference=ACCEPT_AUTOMATIC \
--nat-subnets=psc-nat-subnet
You should note down the Service Attachment URI (selfLink) as you will need it in an upcoming step for the PSC Endpoint configuration. You can obtain it by executing the following in Cloud Shell.
From Cloud Shell
gcloud compute service-attachments describe producer-attachment --region=$region
Sample output
connectionPreference: ACCEPT_AUTOMATIC creationTimestamp: '2026-02-10T07:50:04.250-08:00' description: '' enableProxyProtocol: false fingerprint: xxx id: 'xxx' kind: compute#serviceAttachment name: producer-attachment natSubnets: - https://www.googleapis.com/compute/v1/projects/$project/regions/$region/subnetworks/psc-nat-subnet pscServiceAttachmentId: high: 'xxx' low: 'xxx' reconcileConnections: false region: https://www.googleapis.com/compute/v1/projects/$project/regions/$region selfLink: https://www.googleapis.com/compute/v1/projects/$project/regions/$region/serviceAttachments/producer-attachment targetService: https://www.googleapis.com/compute/v1/projects/$project/regions/$region/forwardingRules/producer-fr
Copy the URI starting from projects
Example: projects/$project/regions/$region/serviceAttachments/producer-attachment
9. Build Consumer Network [Consumer Project]
Create VPC Network
From Cloud Shell
gcloud compute networks create consumer-vpc \
--subnet-mode=custom
Create Subnet
A single subnet will be deployed where we will deploy the PSC endpoint and a test client VM.
From Cloud Shell
gcloud compute networks subnets create consumer-subnet \
--network=consumer-vpc \
--range=10.0.0.0/28 \
--region=$region
Create Network Firewall Policy and Rules
From Cloud Shell
gcloud compute network-firewall-policies create consumer-vpc-policy --global
gcloud compute network-firewall-policies associations create \
--firewall-policy consumer-vpc-policy \
--network consumer-vpc \
--name network-consumer-vpc \
--global-firewall-policy
We'll also configure the consumer-vpc to use IAP for SSH.
From Cloud Shell
gcloud compute network-firewall-policies rules create 1000 \
--action ALLOW \
--firewall-policy consumer-vpc-policy \
--description "SSH with IAP" \
--direction INGRESS \
--src-ip-ranges 35.235.240.0/20 \
--layer4-configs tcp:22 \
--global-firewall-policy
10. Create the Private Service Connect Endpoint [Consumer Project]
Reserve the static IP address that will be used for the PSC endpoint.
From Cloud Shell
gcloud compute addresses create psc-endpoint-ip \
--region=$region \
--subnet=consumer-subnet \
--addresses 10.0.0.2
Create the PSC endpoint using the reserved static IP address and the service attachment URI that we noted earlier.
From Cloud Shell
gcloud compute forwarding-rules create psc-endpoint \ --region=$region \ --network=consumer-vpc \ --address=psc-endpoint-ip \ --target-service-attachment=projects/$project/regions/$region/serviceAttachments/producer-attachment
11. Configure a private DNS zone for the PSC Endpoint [Consumer Project]
Using Cloud DNS, we will configure a private DNS zone with the DNS name myservice.com, to be used with our PSC endpoint.
From Cloud Shell
gcloud dns managed-zones create "consumer-service" \
--dns-name=myservice.com \
--description="consumer service dns" \
--visibility=private \
--networks=consumer-vpc
gcloud dns record-sets transaction start \
--zone="consumer-service"
Create an A record for myservice.com and point it to the PSC endpoint IP address.
From Cloud Shell
gcloud dns record-sets transaction add 10.0.0.2 \ --name=myservice.com \ --ttl=300 \ --type=A \ --zone="consumer-service" gcloud dns record-sets transaction execute \ --zone="consumer-service"
12. Test the PSC Endpoint [Consumer Project]
Create a client VM
From Cloud Shell
gcloud compute instances create consumer-client-vm \
--zone=$zone \
--subnet=consumer-subnet \
--no-address
Test PSC endpoint connectivity
From Cloud Shell
gcloud compute ssh consumer-client-vm \
--zone=$zone \
--command="curl -s myservice.com/index.html"
Expected output
I am a Producer Service.
In the next part of the codelab, we will create and migrate to the new producer service, showcasing the PSC Mutability feature that allows this seamless update without requiring any updates on the consumer side configuration.
13. Create Updated Producer Service [Producer Project]
To update our producer service, we'll deploy a new VM running a similar Apache web server with the message "I am a NEW Producer service." We'll add that VM to a new unmanaged instance group and use that unmanaged instance group as a backend for our new load balancer. Instead of using a Network Passthrough load balancer, we'll update our load balancer to a regional internal Application load balancer.
Update the Network
Create a new subnet that will be used for the proxy-only subnet for the Application load balancer.
From Cloud Shell
gcloud compute networks subnets create lb-proxy-subnet \
--network=producer-vpc \
--range=10.200.0.0/24 \
--region=$region \
--purpose=REGIONAL_MANAGED_PROXY \
--role=ACTIVE
Create a new subnet where our new service will be hosted.
From Cloud Shell
gcloud compute networks subnets create producer-service-new-subnet \
--network=producer-vpc \
--range=10.0.1.0/28 \
--region=$region
Create a new firewall rule that will allow connectivity to our new producer service from the proxy-only subnet.
From Cloud Shell
gcloud compute network-firewall-policies rules create 3001 \
--action ALLOW \
--firewall-policy producer-vpc-policy \
--description "allow access from proxy only subnet" \
--direction INGRESS \
--src-ip-ranges 10.200.0.0/24 \
--layer4-configs tcp:80 \
--global-firewall-policy
Create Instance
From Cloud Shell
gcloud compute instances create new-producer-service-vm \
--network producer-vpc \
--subnet producer-service-new-subnet \
--zone $zone \
--no-address \
--metadata startup-script='#! /bin/bash
sudo apt-get update
sudo apt-get install apache2 -y
a2enmod ssl
sudo a2ensite default-ssl
echo "I am a NEW Producer Service." | \
tee /var/www/html/index.html
systemctl restart apache2'
Create Unmanaged Instance Group
From Cloud Shell
gcloud compute instance-groups unmanaged create producer-new-uig \
--zone=$zone
gcloud compute instance-groups unmanaged add-instances producer-new-uig \
--zone=$zone \
--instances=new-producer-service-vm
gcloud compute instance-groups unmanaged set-named-ports producer-new-uig \
--named-ports=http:80 \
--zone=$zone
Create New Load Balancer Components
Create the backend service. Note that we are using the load balancing scheme INTERNAL_MANAGED because we're updating to an internal application load balancer.
From Cloud Shell
gcloud compute backend-services create producer-new-bes \ --load-balancing-scheme=INTERNAL_MANAGED \ --protocol=http \ --region=$region \ --health-checks=producer-hc \ --health-checks-region=$region
Add the Unmanaged Instance Group as a backend to the backend service.
From Cloud Shell
gcloud compute backend-services add-backend producer-new-bes \ --region=$region \ --instance-group=producer-new-uig \ --instance-group-zone=$zone
Create the URL Map and target HTTP proxies.
From Cloud Shell
gcloud compute url-maps create producer-url-map \ --default-service=producer-new-bes \ --region=$region gcloud compute target-http-proxies create http-proxy \ --url-map=producer-url-map \ --region=$region
Create the static IP address that we will use for our forwarding rule. This will come from the same subnet that we used for the original forwarding rule.
From Cloud Shell
gcloud compute addresses create producer-fr-new-ip\ --region $region \ --subnet producer-fr-subnet \ --addresses 192.168.0.3
Create the forwarding rule.
From Cloud Shell
gcloud compute forwarding-rules create new-producer-fr \ --load-balancing-scheme=INTERNAL_MANAGED \ --network=producer-vpc \ --subnet=producer-fr-subnet \ --address=producer-fr-new-ip \ --ports=80 \ --region=$region \ --target-http-proxy=http-proxy \ --target-http-proxy-region=$region
14. Test the Updated Producer Service [Producer Project]
We'll use the same test VM that we created to test the originally deployed service.
From Cloud Shell
gcloud compute ssh producer-test-vm \
--zone=$zone \
--command="curl -s 192.168.0.3/index.html"
Note that we are calling an updated IP address! 192.168.0.3 is the IP address for our regional internal Application load balancer.
Expected output
I am a NEW Producer Service.
15. Update the Service Attachment [Producer Project]
To better visualize the update to the Service Attachment forwarding rule, we can run a describe on the Service Attachment before and after the change.
From Cloud Shell
gcloud compute service-attachments describe producer-attachment \
--region=$region \
--format="value(targetService)"
Expected output
https://www.googleapis.com/compute/v1/projects/$project/regions/$region/forwardingRules/producer-fr
Update the Service Attachment.
From Cloud Shell
gcloud compute service-attachments update producer-attachment \ --region=$region \ --target-service=projects/$project/regions/$region/forwardingRules/new-producer-fr
Now run the describe again to see the new forwarding rule.
From Cloud Shell
gcloud compute service-attachments describe producer-attachment \
--region=$region \
--format="value(targetService)"
Expected output
https://www.googleapis.com/compute/v1/projects/$project/regions/$region/forwardingRules/new-producer-fr
16. Re-Test the Consumer PSC Connection [Consumer Project]
Log into the same consumer client.
From Cloud Shell
gcloud compute ssh consumer-client-vm \
--zone=$zone \
--command="curl -s myservice.com/index.html"
Expected output
I am a NEW Producer Service.
Congratulations! You've successfully updated a PSC producer service load balancer and it required no configuration changes on the consumer side!
17. Cleanup steps [Consumer Project]
From a single Cloud Shell terminal delete lab components
gcloud dns record-sets delete myservice.com --zone="consumer-service" --type=A -q gcloud dns managed-zones delete "consumer-service" -q gcloud compute forwarding-rules delete psc-endpoint --region=$region -q gcloud compute addresses delete psc-endpoint-ip --region=$region -q gcloud compute instances delete consumer-client-vm --zone=$zone --project $project -q gcloud compute network-firewall-policies rules delete 1000 --firewall-policy consumer-vpc-policy --global-firewall-policy -q gcloud compute network-firewall-policies associations delete --firewall-policy=consumer-vpc-policy --name=network-consumer-vpc --global-firewall-policy -q gcloud compute network-firewall-policies delete consumer-vpc-policy --global -q gcloud compute networks subnets delete consumer-subnet --region $region -q gcloud compute networks delete consumer-vpc -q
18. Cleanup steps [Producer Project]
From a single Cloud Shell terminal delete lab components
gcloud compute service-attachments delete producer-attachment --region=$region -q gcloud compute instances delete producer-test-vm --zone=$zone --project $project -q gcloud compute forwarding-rules delete new-producer-fr --region=$region -q gcloud compute addresses delete producer-fr-new-ip --region $region -q gcloud compute target-http-proxies delete http-proxy --region $region -q gcloud compute url-maps delete producer-url-map --region $region -q gcloud compute backend-services delete producer-new-bes --region $region -q gcloud compute instance-groups unmanaged delete producer-new-uig --zone $zone -q gcloud compute instances delete new-producer-service-vm --zone $zone --project $project -q gcloud compute networks subnets delete producer-service-new-subnet --region $region -q gcloud compute networks subnets delete lb-proxy-subnet --region $region -q gcloud compute forwarding-rules delete producer-fr --region=$region -q gcloud compute addresses delete producer-fr-ip --region $region -q gcloud compute backend-services delete producer-bes --region $region -q gcloud compute health-checks delete producer-hc --region $region -q gcloud compute instance-groups unmanaged delete producer-uig --zone $zone -q gcloud compute instances delete producer-service-vm --zone $zone --project $project -q gcloud compute network-firewall-policies rules delete 4000 --firewall-policy producer-vpc-policy --global-firewall-policy -q gcloud compute network-firewall-policies rules delete 3001 --firewall-policy producer-vpc-policy --global-firewall-policy -q gcloud compute network-firewall-policies rules delete 3000 --firewall-policy producer-vpc-policy --global-firewall-policy -q gcloud compute network-firewall-policies rules delete 2000 --firewall-policy producer-vpc-policy --global-firewall-policy -q gcloud compute network-firewall-policies rules delete 1000 --firewall-policy producer-vpc-policy --global-firewall-policy -q gcloud compute network-firewall-policies associations delete --firewall-policy=producer-vpc-policy --name=network-producer-vpc --global-firewall-policy -q gcloud compute network-firewall-policies delete producer-vpc-policy --global -q gcloud compute routers nats delete $region-nat --router=$region-cr --region=$region -q gcloud compute routers delete $region-cr --region=$region -q gcloud compute networks subnets delete psc-nat-subnet --region=$region -q gcloud compute networks subnets delete producer-fr-subnet --region=$region -q gcloud compute networks subnets delete producer-service-subnet --region=$region -q gcloud compute networks delete producer-vpc -q
19. Congratulations!
Congratulations for completing the codelab.
What we've covered
- Create a simple Apache web service exposed as a PSC producer service.
- Create a PSC endpoint.
- Create a Cloud DNS private zone for the consumer service calls.
- Update the forwarding rule associated with the service attachment using PSC Mutability.