1. Introduction
Private Service Connect lets a service producer offer services privately to a service consumer. Private Service Connect offers the following benefits:
- A service producer VPC network can support more than one service consumer.
- Each consumer connects to an internal IP address that they define. Private Service Connect performs network address translation (NAT) to route the request to the service producer.
Figure 2. Private Service Connect uses endpoints and service attachments to let service consumers send traffic from the consumer's VPC network to services in the service producer's VPC network (click to enlarge).
What you'll learn
- Private service connect benefits
- Key concepts for service consumers
- Key concepts for service producers
- Create a producer environment
- Expose service (producer environment) through a service attachment
- Create a consumer environment
- Create a forwarding rule in the consumer network
- Validate consumer access
- Enable policy access control
- Use a egress firewall rule to block access to a consumer forwarding rule
What you'll need
- Knowledge of deploying GKE Clusters and services
- Knowledge of Internal load balancers
- Ability to create VPCs in two projects
- Ability to create GKE Cluster
2. Private service connect benefits
With PSC, you have several benefits compared to using VPC Peering:
Better control of private IP space
- As a service consumer, you can control the private IP address that is used to connect to the managed service you would like to access.
- As a service consumer, you do not need to worry about reserving private IP address ranges for backend services that are consumed in your VPC. You only need to choose an IP address from your own subnet to connect to the producer services.
- As a service producer, you can choose to deploy a multi-tenant model, where your VPC contains services that serve multiple consumer VPCs. The consumers having overlapping subnet ranges are no longer a problem.
- As a service provider, you can scale your service to as many VM instances as required, without needing to contact your consumer for more IP addresses.
Improved security and isolation
- As a service consumer, only you can initiate communication to the service producer. This uni-directional connectivity drastically simplifies firewall configuration but also reduces risk from rogue traffic coming from the service producer.
- As a service producer, you do not need to change your firewall rules based on the subnet ranges in the consumer's VPC. You can simply create firewall rules for the NAT IP address range configured for your service.
Better Scalability
- PSC enables highly-scalable design by supporting thousands of Consumers, and allows Service Producers to offer highly scalable multi-tenant or single-tenant services.
- As a service consumer using private service connect, you can create resources as required in your VPC. The scale of this is not affected by the number of such resources created in the producer VPC.
3. Key concepts for service consumers
You can use Private Service Connect endpoints to consume services that are outside of your VPC network. Service consumers create Private Service Connect endpoints that connect to a target service.
Endpoints
You use Private Service Connect endpoints to connect to a target service. Endpoints have an internal IP address in your VPC network and are based on the forwarding rule resource.
You send traffic to the endpoint, which forwards it to targets outside of your VPC network.
Targets
Private Service Connect endpoints have a target, which is the service you want to connect to:
- An API bundle:
- All APIs: most Google APIs
- VPC-SC: APIs that VPC Service Controls supports
- A published service in another VPC network. This service can be managed by your own organization or a third party.
Published service
To connect your endpoint to a service producer's service, you need the service attachment for the service. The service attachment URI has this format: projects/SERVICE_PROJECT/regions/REGION/serviceAttachments/SERVICE_NAME
4. Key concepts for service producers
To make a service available to consumers, you create one or more dedicated subnets to use for network address translation (NAT) of consumer IP addresses. You then create a service attachment which refers to those subnets.
Private Service Connect subnets
To expose a service, the service producer first creates one or more subnets with purpose Private Service Connect.
When a request is sent from a consumer VPC network, the consumer's source IP address is translated using source NAT (SNAT) to an IP address selected from one of the Private Service Connect subnets.
If you want to retain the consumer connection IP address information, see Viewing consumer connection information.
These subnets cannot be used for resources such as VM instances or forwarding rules. The subnets are used only to provide IP addresses for SNAT of incoming consumer connections.
The Private Service Connect subnet must contain at least one IP address for every 63 consumer VMs so that each consumer VM is allocated 1,024 source tuples for network address translation.
The minimum size for a Private Service Connect subnet is /24.
Service attachments
Service producers expose their service through a service attachment.
- To expose a service, a service producer creates a service attachment that refers to the service's load balancer forwarding rule.
- To access a service, a service consumer creates an endpoint that refers to the service attachment.
Connection preferences
When you create a service, you choose how to make it available. There are two options:
- Automatically accept connections for all projects - any service consumer can configure an endpoint and connect to the service automatically.
- Accept connections for selected projects - service consumers configure an endpoint to connect to the service and the service producer accepts or rejects the connection requests.
Requirements and limitations
- Limitations for Private Service Connect apply.
- You can create a service attachment in GKE versions 1.21.4-gke.300 and later.
- You cannot use the same subnet in multiple service attachment configurations.
- You must create a GKE service that uses an internal TCP/UDP load balancer.
5. Test Environment
Consumer network consists of a static IP address used to originate requests to the service producer, in addition to the target-service-attachment that maps to the producer's service attachment (published service).
Now, let's take a look at the producers network. Notice how the producers network does not have a mapping to the consumers network, instead, the producer network contains a service attachment (published service) that is used by the consumer for services. The producer's service attachment is exposed by a GKE ingress L4 ILB (published service) enabling communication to the GKE Pods & associated applications.
The NAT subnet is used when a request is sent from a consumer VPC network, the consumer's source IP address is translated using source NAT (SNAT) to an IP address selected from one of the Private Service Connect subnets.
These subnets cannot be used for resources such as VM instances or forwarding rules. The subnets are used only to provide IP addresses for SNAT of incoming consumer connections.
To find out more about L4ILB for GKE Private Service Connect and obtain direct access to content used to make this lab reference the following.
Self-paced environment setup
- Sign-in to the Google Cloud Console and create a new project or reuse an existing one. If you don't already have a Gmail or Google Workspace account, you must create one.
- The Project name is the display name for this project's participants. It is a character string not used by Google APIs, and you can update it at any time.
- The Project ID must be unique across all Google Cloud projects and is immutable (cannot be changed after it has been set). The Cloud Console auto-generates a unique string; usually you don't care what it is. In most codelabs, you'll need to reference the Project ID (and it is typically identified as
PROJECT_ID
), so if you don't like it, generate another random one, or, you can try your own and see if it's available. Then it's "frozen" after the project is created. - There is a third value, a Project Number which some APIs use. Learn more about all three of these values in the documentation.
- Next, you'll need to enable billing in the Cloud Console in order to use Cloud resources/APIs. Running through this codelab shouldn't cost much, if anything at all. To shut down resources so you don't incur billing beyond this tutorial, follow any "clean-up" instructions found at the end of the codelab. New users of Google Cloud are eligible for the $300 USD Free Trial program.
Start Cloud Shell
While Google Cloud can be operated remotely from your laptop, in this codelab you will be using Google Cloud Shell, a command line environment running in the Cloud.
From the GCP Console click the Cloud Shell icon on the top right toolbar:
It should only take a few moments to provision and connect to the environment. When it is finished, you should see something like this:
This virtual machine is loaded with all the development tools you'll need. It offers a persistent 5GB home directory, and runs on Google Cloud, greatly enhancing network performance and authentication. All of your work in this lab can be done with simply a browser.
6. Before you begin
Codelab requires two projects, although not a requirement for PSC. Note the references to support single or multiple projects.
Single Project - Update project to support producer and consumer network
Inside Cloud Shell, make sure that your project id is set up
gcloud config list project gcloud config set project [YOUR-PROJECT-NAME] prodproject=YOUR-PROJECT-NAME consumerproject=YOUR-PROJECT-NAME echo $prodproject echo $consumerproject
Multiple projects - Update project to support producer network
Inside Cloud Shell, make sure that your project id is set up
gcloud config list project gcloud config set project [YOUR-PROJECT-NAME] prodproject=YOUR-PROJECT-NAME echo $prodproject
Note the following coloring code convention:
7. Create Producers VPC network
VPC Network
From Cloud Shell
gcloud compute networks create gke-producer-l4-vpc --project=$prodproject --subnet-mode=custom
Create GKE Cluster subnet
From Cloud Shell
gcloud compute networks subnets create node-subnet1 --project=$prodproject --range=192.168.10.0/24 --network=gke-producer-l4-vpc --region=us-central1 --secondary-range=pod=10.10.10.0/24,service=10.10.20.0/24 --enable-private-ip-google-access
Create GKE Cluster
From Cloud Shell
gcloud container clusters create gke-psc-l4 \ --release-channel=rapid \ --enable-ip-alias \ --zone=us-central1-a \ --network gke-producer-l4-vpc \ --num-nodes 1 \ --subnetwork node-subnet1 \ --cluster-secondary-range-name pod \ --services-secondary-range-name service
Create a subnet for Private Service Connect (NAT Subnet)
You must create one or more dedicated subnets for use with Private Service Connect. If you're using the Google Cloud Console to publish a service, you can create the subnets during that procedure.
For information about Private Service Connect subnets, see Private Service Connect subnets.
From Cloud Shell
gcloud beta compute networks subnets create gke-nat-subnet \ --project $prodproject \ --network gke-producer-l4-vpc \ --region us-central1 \ --range 100.100.10.0/24 \ --purpose PRIVATE_SERVICE_CONNECT
8. Deploy a workload and services
The following manifest describes a Deployment that runs a sample web application container image. Save the manifest as my-deployment.yaml from cloud shell
apiVersion: apps/v1 kind: Deployment metadata: name: psc-ilb spec: replicas: 3 selector: matchLabels: app: psc-ilb template: metadata: labels: app: psc-ilb spec: containers: - name: whereami image: gcr.io/google-samples/whereami:v1.2.1 ports: - name: http containerPort: 8080 readinessProbe: httpGet: path: /healthz port: 8080 scheme: HTTP initialDelaySeconds: 5 timeoutSeconds: 1
Apply the manifest to your cluster from cloud shell
kubectl apply -f my-deployment.yaml
Create a service
The following manifest describes a service that creates an internal TCP/UDP load balancer on TCP port 8080. Save the manifest as my-service.yaml from cloud shell.
apiVersion: v1 kind: Service metadata: name: gke-l4-psc annotations: networking.gke.io/load-balancer-type: "Internal" spec: type: LoadBalancer selector: app: psc-ilb ports: - port: 80 targetPort: 8080 protocol: TCP
Apply the manifest to your cluster from cloud shell
kubectl apply -f my-service.yaml
Create ServiceAttachment
The following manifest describes a ServiceAttachment that exposes the service that you created to service consumers. Save the manifest as my-psc.yaml from cloud shell.
apiVersion: networking.gke.io/v1beta1 kind: ServiceAttachment metadata: name: emoji-sa namespace: default spec: connectionPreference: ACCEPT_AUTOMATIC natSubnets: - gke-nat-subnet proxyProtocol: false resourceRef: kind: Service name: gke-l4-psc
Apply the manifest to your cluster from cloud shell
kubectl apply -f my-psc.yaml
The ServiceAttachment has the following fields:
- connectionPreference: the connection preference that determines how customers connect to the service. You can either use automatic project approval using ACCEPT_AUTOMATIC or explicit project approval using ACCEPT_MANUAL. For more information, see Publishing services using Private Service Connect.
- natSubnets: a list of subnetwork resource names to use for the service attachment.
- proxyProtocol: when set to true, the consumer source IP and Private Service Connect connection ID are available in the requests. This field is optional and defaults to false if not provided.
- consumerAllowList: the list of consumer projects that are allowed to connect to the ServiceAttachment. This field can only be used when connectionPreference is ACCEPT_MANUAL. For more information about this field and other options see Publishing services using Private Service Connect.
Producer Validation
Viewing the Service Attachment details
You can view the details of a ServiceAttachment using the following command from cloud shell
kubectl describe serviceattachment emoji-sa
View the GKE L4 ILB
From cloud console, navigate to Network Services → Load Balancing → Frontends
Identify the frontend IP address that aligns into the previously defined node subnet 192.168.10.0/24, note screenshot below, your IP Address may differ.
View the Published Service
From cloud console, navigate to Network Services → Private Service Connect → Published Services
Identify the service with the network used in the lab, gke-producer-l4-vpc, note screenshot below, although your Service and Target values may differ
Click on the service name that advances you to the screen below, note the service attachment details populated in the Basic Info. Also, note ‘Connected Projects' is empty since the consumer has yet to register with the service. ACCEPT and REJECT will remain greyed out since Connection preference is set to "ACCEPT_AUTOMATICALLY"', this option can be changed at any time to "ACCEPT_MANUAL" by modifying the the service attachment yaml (my-psc.yaml).
9. Create Consumers VPC network
In the following section the consumer VPC is configured in a separate project. Communication between the consumer and producer network is accomplished through the service attachment defined in the consumers network.
Codelab requires two projects, although not a requirement for PSC. Note the references to support single or multiple projects.
Single Project - Update project to support producer and consumer network
Inside Cloud Shell, make sure that your project id is set up
gcloud config list project gcloud config set project [YOUR-PROJECT-NAME] consumerproject=YOUR-PROJECT-NAME prodproject=YOUR-PROJECT-NAME echo $prodproject echo $consumerproject
Multiple projects - Update project to support consumer a network
Inside Cloud Shell, make sure that your project id is set up
gcloud config list project gcloud config set project [YOUR-PROJECT-NAME] consumerproject=YOUR-PROJECT-NAME echo $consumerproject
VPC Network
From Cloud Shell
gcloud compute networks create vpc-demo-consumer --project=$consumerproject --subnet-mode=custom
Create a subnet for PSC
From Cloud Shell
gcloud compute networks subnets create consumer-subnet --project=$consumerproject --range=10.0.60.0/24 --network=vpc-demo-consumer --region=us-central1
Create a subnet for VM instances
From Cloud Shell
gcloud compute networks subnets create consumer-subnet-vm --project=$consumerproject --range=10.0.70.0/24 --network=vpc-demo-consumer --region=us-central1
Create a static IP address to access the published service
From Cloud Shell
gcloud compute addresses create vpc-consumer-psc --region=us-central1 --subnet=consumer-subnet --addresses 10.0.60.100
Create Firewall Rules
To allow IAP to connect to your VM instances, create a firewall rule that:
- Applies to all VM instances that you want to be accessible by using IAP.
- Allows ingress traffic from the IP range 35.235.240.0/20. This range contains all IP addresses that IAP uses for TCP forwarding
From Cloud Shell
gcloud compute firewall-rules create psclab-iap-consumer --network vpc-demo-consumer --allow tcp:22 --source-ranges=35.235.240.0/20 --enable-logging
Although not required for PSC create a egress firewall rule to monitor consumer PSC traffic to the producers service attachment
gcloud compute --project=$consumerproject firewall-rules create vpc-consumer-psc --direction=EGRESS --priority=1000 --network=vpc-demo-consumer --action=ALLOW --rules=all --destination-ranges=10.0.60.0/24 --enable-logging
10. Create consumer test instance 1
From Cloud Shell
gcloud compute instances create consumer-instance-1 --zone=us-central1-a --machine-type=e2-micro --private-network-ip=10.0.70.10 --no-address --subnet=consumer-subnet-vm --tags=google1 --image-family=debian-10 --image-project=debian-cloud
11. Create consumer test instance 2
From Cloud Shell
gcloud compute instances create consumer-instance-2 --zone=us-central1-a --machine-type=e2-micro --private-network-ip=10.0.70.20 --no-address --subnet=consumer-subnet-vm --tags=google2 --image-family=debian-10 --image-project=debian-cloud
12. Create a service attachment
In a previous step, you copied the Producer Service Attachment string in a safe place, let's insert the stored value into the "target-service-attachment" field.
From Cloud Shell
gcloud compute forwarding-rules create vpc-consumer-psc-fr --region=us-central1 --network=vpc-demo-consumer --address=vpc-consumer-psc --target-service-attachment=yoursavedproducerserviceattachment
13. Validation - Consumer
We will use CURL & firewall logs to validate consumer and producer communication.
Within the Consumer's project the static IP addresses are used to originate communication to the Producer. This mapping of static IP address to Consumer forwarding rule is validated by performing the following syntax.
From the Consumer VPCs Cloud use shell identify the forwarding rule and static IP
gcloud compute forwarding-rules describe vpc-consumer-psc-fr --region us-central1
Output below, we will use 10.0.60.100 to reach the producer in a later step
IPAddress: 10.0.60.100 creationTimestamp: '2021-09-30T21:13:54.124-07:00' id: '3564572805904938477' kind: compute#forwardingRule labelFingerprint: 42WmSpB8rSM= name: vpc-consumer-psc-fr network: https://www.googleapis.com/compute/v1/projects/deepakmichaelstage/global/networks/vpc-demo-consumer networkTier: PREMIUM pscConnectionId: '36583161500548196' pscConnectionStatus: ACCEPTED
View the Connected Service
From cloud console, navigate to Network Services → Private Service Connect → Connected Endpoints & view the newly created endpoint
Lets log into consumer-instance-1 and test access to the Producer Published Service
From cloud shell open a new tab by clicking the +
From cloud shell perform the following:
gcloud config list project gcloud config set project [YOUR-PROJECT-NAME] projectname=YOUR-PROJECT-NAME echo $projectname gcloud compute ssh --zone "us-central1-a" "consumer-instance-1" --project "$projectname"
Once logged into consumer-instance-1 instance perform a curl against the forwarding rule IP Address 10.0.60.100
From cloud shell perform the following:
user@consumer-instance-1:~$ curl 10.0.60.100
Example output
user@consumer-instance-1:~$ curl 10.0.60.100 { "cluster_name": "gke-psc-l4", "host_header": "10.0.60.100", "node_name": "gke-gke-psc-l4-default-pool-f2c6e301-vnlz.c.prodprojectid.internal", "pod_name": "psc-ilb-588887dfdb-w7tbr", "pod_name_emoji": "🤷", "project_id": "prodorijectid", "timestamp": "2021-10-01T17:43:37", "zone": "us-central1-a"
Lets log into consumer-instance-2 and test access to the Producer Published Service
From cloud shell open a new tab by clicking the +
From cloud shell perform the following:
gcloud config list project gcloud config set project [YOUR-PROJECT-NAME] projectname=YOUR-PROJECT-NAME echo $projectname gcloud compute ssh --zone "us-central1-a" "consumer-instance-2" --project "$projectname"
From cloud shell perform the following:
user@consumer-instance-2:~$ curl 10.0.60.100
Example output
deepakmichael@consumer-instance-2:~$ curl 10.0.60.100 { "cluster_name": "gke-psc-l4", "host_header": "10.0.60.100", "node_name": "gke-gke-psc-l4-default-pool-f2c6e301-vnlz.c.prodproject.internal", "pod_name": "psc-ilb-588887dfdb-4jdql", "pod_name_emoji": "🧑🏿", "project_id": "prodproject", "timestamp": "2021-10-01T17:49:51", "zone": "us-central1-a"
14. Firewall logging - Allocated Validation
Using Logs Explorer validate firewall rule "vpc-consumner-psc" is capturing the flow between the VM's instance and static IP
- From Cloud Console, Identify Operations Logging → Log Explorer
- In the Query field update the entry below with yourconsumerproject and select "Run Query"
logName:(projects/yourconsumerprojectID/logs/compute.googleapis.com%2Ffirewall) AND jsonPayload.rule_details.reference:("network:vpc-demo-consumer/firewall:vpc-consumer-psc")
- Query results provide the following per screenshot provided
- Expand the log (jsonPayload → Connection) and identify the output provided below. Take note of the dest_ip: 10.0.60.100 is the STATIC TCP IP used to access the Producer Service and src_ip: 10.0.70.10 or 10.0.70.20 are the VM instance IP address(s). Disposition is Allowed.
15. Validation - Producer
From the Producers project verify the service attachment is connected successfully. Navigate to Network Services → Private Service Connect → Published Services
Click on the service reveals your connected consumer project and status as illustrated below
16. Restrict access to a Published Service
Thus far we have confirmed both instances have access to the Published Services, let's create an Egress Firewall Rule to deny consumer-instance-2 access to the Published Service.
By default GCP allows all Egress yet denies all Ingress traffic. In the following steps we will create a firewall rule based on a previously defined networking tag ‘google2' used when creating consumer-instance-2 to deny access to the Published Service.
Open a new cloud shell tab by clicking on the + execute the following firewall rule in cloud shell
gcloud config list project gcloud config set project [YOUR-PROJECT-NAME] projectname=YOUR-PROJECT-NAME echo $projectname gcloud compute --project=$projectname firewall-rules create psc-endpoint-deny-egress --direction=EGRESS --priority=999 --network=vpc-demo-consumer --action=DENY --rules=all --destination-ranges=10.0.60.100/32 --target-tags=google2 --enable-logging
Now, let's test if consumer-instance-2 is able to access the Published Service. If your session timed out you will need to open a new cloud shell + and log into the VM as detailed below.
gcloud config list project gcloud config set project [YOUR-PROJECT-NAME] projectname=YOUR-PROJECT-NAME echo $projectname gcloud compute ssh --zone "us-central1-a" "consumer-instance-2" --project "$projectname"
From cloud shell perform the following:
user@consumer-instance-2:~$ curl 10.0.60.100
Example output
user@consumer-instance-2:~$ curl 10.0.60.100 curl: (7) Failed to connect to 10.0.60.100 port 80: Connection timed out
Firewall logging - Denied Validation
Using Logs Explorer validate firewall rule "psc-endpoint-deny-egress" is capturing the flow between the VM instance and static IP
- From Cloud Console, Identify Operations Logging → Log Explorer
- In the Query field update the entry below with yourconsumerproject and select "Run Query"
logName:(projects/yourconsumerprojectID/logs/compute.googleapis.com%2Ffirewall) AND jsonPayload.rule_details.reference:("network:vpc-demo-consumer/firewall:psc-endpoint-deny-egress")
- Query results provide the following per screenshot provided
- Expand the log and identify the output provided below. Take note of the dest_ip: 10.0.60.100 is the STATIC TCP IP and src_ip: 10.0.70.10 or 10.0.70.20 are the VM instance IP address(s). Disposition is Denied.
17. Cleanup steps
Producer network clean up steps
From a single cloud shell in the Producer project terminal delete lab components
gcloud container clusters delete gke-psc-l4 --region us-central1-a --quiet gcloud compute networks subnets delete gke-nat-subnet --region=us-central1 --quiet gcloud compute networks subnets delete node-subnet1 --region=us-central1 --quiet gcloud compute networks delete gke-producer-l4-vpc --quiet
Consumer network clean up steps
From a single cloud shell in the Consumer project terminal delete lab components
gcloud compute instances delete consumer-instance-1 --zone=us-central1-a --quiet gcloud compute instances delete consumer-instance-2 --zone=us-central1-a --quiet gcloud compute forwarding-rules delete vpc-consumer-psc-fr --region=us-central1 --quiet gcloud compute addresses delete vpc-consumer-psc --region=us-central1 --quiet gcloud compute firewall-rules delete psclab-iap-consumer --quiet gcloud compute networks subnets delete consumer-subnet --region=us-central1 --quiet gcloud compute networks subnets delete consumer-subnet-vm --region=us-central1 --quiet gcloud compute firewall-rules delete vpc-consumer-psc --quiet gcloud compute firewall-rules delete psc-endpoint-deny-egress --quiet gcloud compute networks delete vpc-demo-consumer --quiet
18. Congratulations!
Congratulations for completing the codelab.
What we've covered
- Private service connect benefits
- Key concepts for service consumers
- Key concepts for service producers
- Create a producer environment
- Expose service (producer environment) through a service attachment
- Create a consumer environment
- Create a forwarding rule in the consumer network
- Validate consumer access
- Enable policy access control
- Used an egress firewall rule to block access to a consumer forwarding rule