1. Introduction
Cloud Secure Web Proxy
Cloud SWP is a cloud-first service that provides a secure web proxy to help you secure egress web traffic (HTTP/S). You configure your clients to explicitly use Cloud SWP as a proxy. The web requests can originate from the following sources:
- Virtual machine (VM) instances
- Containers
- A serverless environment that uses a serverless connector
- Workloads across VPC Peering
- Workloads outside of Google Cloud connected by Cloud VPN or Cloud Interconnect
Cloud SWP enables flexible and granular policies based on cloud-first identities and web applications.
Benefits
Below are some examples of benefits that Cloud SWP could provide an organization:
Migration to Google Cloud
Cloud SWP helps you migrate to Google Cloud while keeping your existing security policies and requirements for egress web traffic. You can avoid using third-party solutions that require another management console or manually editing configuration files.
Access to trusted external web services
Cloud SWP lets you apply granular access policies to your egress web traffic so that you can secure your network. You create and identify workload or application identities and then apply policies.
Monitored access to untrusted web services
You can use Cloud SWP to provide monitored access to untrusted web services. Cloud SWP identifies traffic that doesn't conform to policy and logs it to Cloud Logging (Logging). You can then monitor internet usage, discover threats to your network, and respond to threats.
Granular policy controls for Google APIs
You can use Cloud SWP to provide granular policies for Google APIs. For example, you can set bucket/object level policies leveraging Common Expression Language (CEL).
Supported Features
Cloud SWP supports the following features:
Explicit proxy service
Clients must be explicitly configured to use the proxy server. The Cloud SWP proxy isolates clients from the internet by creating new TCP connections on the client's behalf.
Autoscaling Cloud SWP Envoy proxies
Supports automatically adjusting the Envoy proxy pool size and the pool's capacity in a region, which enables consistent performance during high-demand periods at the lowest cost.
Modular egress access policies
Cloud SWP specifically supports the following egress policies:
- Source-identity based on secure tags, service accounts, or IP addresses.
- Destinations based on URLs, hostnames.
- Requests based on methods, headers, or URLs. URLs can be specified by using lists, wildcards, or patterns.
- End-to-end encryption: Client-proxy tunnels might transit over TLS. Cloud SWP also supports HTTP/S CONNECT for client-initiated, end-to-end TLS connections to the destination server.
Simplified Cloud NAT Integration
Cloud NAT automatically provisions additional public IP addresses when the set of proxies that serve Cloud SWP traffic increases.
Manual static public IP addresses is also an option for those wanting to have known egress IPs.
Cloud Audit Logs and Google Cloud's operations suite integration
Cloud Audit Logs and Google Cloud's operations suite record administrative activities and access requests for Cloud SWP-related resources. They also record metrics and transaction logs for requests handled by the proxy.
TLS Inspection
Secure Web Proxy offers a TLS inspection service that lets you intercept the TLS traffic, inspect the encrypted request, and enforce security policies.
- Tight integration with Certificate Authority Service (CAS), which is a highly available and scalable repository for private CAs.
- The ability to use your own root of trust if required. You can also use an existing root CA to sign for subordinate CAs held by CAS. If you prefer, you can generate a new root certificate within CAS.
- Granular decryption criteria by using SessionMatcher and ApplicationMatcher within Secure Web Proxy policy rules. This criteria includes matching hosts present in URL lists, regular expressions, IP address ranges, and similar expressions. If required, criteria can be combined with boolean expressions.
- Each Secure Web Proxy policy can be configured with its own TLS inspection policy and CA pool. Alternatively, multiple Secure Web Proxy policies can share a single TLS inspection policy.
What you'll learn
- How to deploy and manage Cloud SWP.
What you'll need
- Knowledge of deploying instances and configuring networking components
- VPC Firewall Configuration Knowledge
2. Test Environment
This codelab will leverage a single VPC. A compute resource in this environment will egress using the Cloud SWP as seen in the diagram below.
In this lab, we will have 2 workload VMs.
Client A will be configured to send all HTTP/HTTPS requests to the Cloud SWP.
Client B will NOT be configured to explicitly send HTTP/HTTPS requests to the Cloud SWP, but instead leverage Cloud NAT for internet bound traffic.
3. Before you begin
Codelab requires a single project.
Inside Cloud Shell, make sure that your project id is set up
export project_id=`gcloud config list --format="value(core.project)"` export project_number=`gcloud projects describe $project_id --format="value(projectNumber)"` export region=us-west1 export zone=us-west1-a export prefix=codelab-swp export member="serviceAccount:service-$project_number@gcp-sa-networksecurity.iam.gserviceaccount.com"
4. Enable APIs
Enable the APIs to use the products
gcloud services enable networksecurity.googleapis.com gcloud services enable certificatemanager.googleapis.com gcloud services enable networkservices.googleapis.com
5. Create VPC network, subnet, and proxy-only subnet
VPC Network
Create codelab-swp-vpc VPC:
gcloud compute networks create $prefix-vpc --subnet-mode=custom
Subnet
Create the respective subnets in the selected region:
gcloud compute networks subnets create $prefix-vpc-subnet \ --range=10.10.10.0/24 --network=$prefix-vpc --region=$region
Proxy-only subnet
Create a proxy-only subnet in the selected region:
gcloud compute networks subnets create $prefix-proxy-only-subnet --purpose=REGIONAL_MANAGED_PROXY --role=ACTIVE --region=$region --network=$prefix-vpc --range=172.16.0.0/23
6. Create Firewall Rules
To allow IAP to connect to your VM instances, create a firewall rule that:
- Applies to all VM instances that you want to be accessible by using IAP.
- Allows ingress traffic from the IP range 35.235.240.0/20. This range contains all IP addresses that IAP uses for TCP forwarding.
From cloudshell:
gcloud compute firewall-rules create $prefix-allow-iap-proxy \ --direction=INGRESS \ --priority=1000 \ --network=$prefix-vpc \ --action=ALLOW \ --rules=tcp:22 \ --source-ranges=35.235.240.0/20
7. Create Cloud Router & Cloud NAT
Create Cloud Router for Cloud NAT.
gcloud compute routers create ${prefix}-cr \
--region=$region \
--network=${prefix}-vpc
Create Cloud NAT Gateway for Client B.
gcloud compute routers nats create $prefix-nat-gw-$region \ --router=$prefix-cr \ --router-region=$region \ --auto-allocate-nat-external-ips \ --nat-all-subnet-ip-ranges
8. Create a Gateway Security Policy
Create a yaml file which contains relevant information for the policy:
cat > /tmp/policy.yaml << EOF
description: Policy to allow .com traffic, then (/index.html), and finally TLS.
name: projects/${project_id}/locations/${region}/gatewaySecurityPolicies/${prefix}-policy
EOF
Run the gcloud command to create the policy from the yaml file:
gcloud network-security gateway-security-policies import ${prefix}-policy --source=/tmp/policy.yaml --location=${region}
9. Create a Gateway Security Policy Rule
Create a yaml file which contains the rules. These rules are represented in Common Expression Language (CEL). This lab will use a simple rule that will allow traffic to .com domains and block all others:
cat > /tmp/rule-com.yaml << EOF
name: projects/${project_id}/locations/${region}/gatewaySecurityPolicies/${prefix}-policy/rules/rule-com
enabled: true
priority: 1
description: Allow .com traffic
basicProfile: ALLOW
sessionMatcher: host().endsWith('com')
EOF
Now we can bind the rule to the gateway security policy:
gcloud network-security gateway-security-policies rules import rule-com --source=/tmp/rule-com.yaml --location=${region} --gateway-security-policy=${prefix}-policy
10. Create a certificate and upload to Cloud Certificate Manager
Create a certificate to terminate workload traffic:
openssl req -x509 -newkey rsa:2048 -keyout /tmp/key.pem -out /tmp/cert.pem -days 365 -subj '/CN=www.codelab-swp.com' -nodes -addext \ "subjectAltName = DNS:www.codelab-swp.com"
Upload the certificate to Cloud Certificate Manager so the SWP could reference it in the security gateway policy.
gcloud certificate-manager certificates create ${prefix}-cert --location=${region} --private-key-file=/tmp/key.pem --certificate-file=/tmp/cert.pem
11. Create the SWP Gateway
Create the yaml file for SWP Gateway to reference the previous information such as certificate, gateway security policy, network, and subnet.
cat > /tmp/gateway.yaml << EOF
name: projects/${project_id}/locations/${region}/gateways/${prefix}-gateway
type: SECURE_WEB_GATEWAY
addresses: [10.10.10.50]
ports: [443]
certificateUrls: [projects/${project_id}/locations/${region}/certificates/${prefix}-cert]
gatewaySecurityPolicy: projects/${project_id}/locations/${region}/gatewaySecurityPolicies/${prefix}-policy
network: projects/${project_id}/global/networks/${prefix}-vpc
subnetwork: projects/${project_id}/regions/${region}/subnetworks/${prefix}-vpc-subnet
EOF
Create the gateway:
gcloud network-services gateways import ${prefix}-swp --source=/tmp/gateway.yaml --location=${region}
Confirm the gateway has been created:
gcloud network-services gateways describe ${prefix}-swp --location ${region}
12. Create Compute Instances
Since Cloud SWP is an explicit proxy, we need to explicitly specify the proxy IP for workload traffic. Compute instance clientA will have the environment variable set. ClientB will not.
Create the compute instances ClientA and ClientB:
gcloud compute instances create clienta \ --subnet=$prefix-vpc-subnet \ --no-address \ --private-network-ip=10.10.10.10 \ --zone $zone \ --metadata startup-script='#! /bin/bash apt-get update sudo echo http_proxy=https://10.10.10.50:443/ >> /etc/environment sudo echo https_proxy=https://10.10.10.50:443/ >> /etc/environment '
gcloud compute instances create clientb \ --subnet=$prefix-vpc-subnet \ --no-address \ --private-network-ip=10.10.10.200 \ --zone $zone \ --metadata startup-script='#! /bin/bash apt-get update '
13. Testing Session Matching
SSH into "clienta" compute VM recently created. This VM has the environmental variable set to use Cloud SWP.
From cloudshell:
gcloud compute ssh clienta --zone=$zone --tunnel-through-iap
Run some web queries to validate functionality. We require –proxy-insecure because we created a self-signed certificate for this lab:
curl https://google.com --proxy-insecure
Expected output:
davidtu@clienta:~$ curl https://google.com --proxy-insecure <HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8"> <TITLE>301 Moved</TITLE></HEAD><BODY> <H1>301 Moved</H1> The document has moved <A HREF="https://www.google.com/">here</A>. </BODY></HTML>
As you can see, the request was "successful". It is expected that we see a 301 redirect since the website is redirecting to https://www.google.com.
Running the following command would provide verbose logs with details about the connection:
curl https://google.com --proxy-insecure -v
Highlighting some outputs to show the proxy connection details, certificates, and destination.
davidtu@clienta:~$ curl https://google.com --proxy-insecure -v * Uses proxy env variable https_proxy == 'https://10.10.10.50:443/' * Trying 10.10.10.50:443... * Connected to 10.10.10.50 (10.10.10.50) port 443 (#0) * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: /etc/ssl/certs/ca-certificates.crt * CApath: /etc/ssl/certs * TLSv1.3 (OUT), TLS handshake, Client hello (1): * TLSv1.3 (IN), TLS handshake, Server hello (2): * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8): * TLSv1.3 (IN), TLS handshake, Certificate (11): * TLSv1.3 (IN), TLS handshake, CERT verify (15): * TLSv1.3 (IN), TLS handshake, Finished (20): * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.3 (OUT), TLS handshake, Finished (20): * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 * ALPN, server accepted to use http/1.1 * Proxy certificate: * subject: CN=www.codelab-swp.com * start date: Dec 12 17:16:35 2022 GMT * expire date: Dec 12 17:16:35 2023 GMT * issuer: CN=www.codelab-swp.com * SSL certificate verify result: self signed certificate (18), continuing anyway. * allocate connect buffer! * Establish HTTP proxy tunnel to google.com:443 > CONNECT google.com:443 HTTP/1.1 > Host: google.com:443 > User-Agent: curl/7.74.0 > Proxy-Connection: Keep-Alive > * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): < HTTP/1.1 200 OK < date: Mon, 12 Dec 2022 19:22:04 GMT < * Proxy replied 200 to CONNECT request * CONNECT phase completed! ...
Feel free to try other .com domains to verify functionality.
Now let's try some other non-.com domains to verify default blocking behavior:
curl https://wikipedia.org --proxy-insecure
Expected output:
curl: (56) Received HTTP code 403 from proxy after CONNECT
Similarly, look at the verbose output logging and confirm Cloud SWP is blocking this traffic:
curl https://wikipedia.org --proxy-insecure -v
davidtu@clienta:~$ curl https://wikipedia.org --proxy-insecure -v * Uses proxy env variable https_proxy == 'https://10.10.10.50:443/' * Trying 10.10.10.50:443... * Connected to 10.10.10.50 (10.10.10.50) port 443 (#0) * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: /etc/ssl/certs/ca-certificates.crt * CApath: /etc/ssl/certs * TLSv1.3 (OUT), TLS handshake, Client hello (1): * TLSv1.3 (IN), TLS handshake, Server hello (2): * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8): * TLSv1.3 (IN), TLS handshake, Certificate (11): * TLSv1.3 (IN), TLS handshake, CERT verify (15): * TLSv1.3 (IN), TLS handshake, Finished (20): * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.3 (OUT), TLS handshake, Finished (20): * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 * ALPN, server accepted to use http/1.1 * Proxy certificate: * subject: CN=www.codelab-swp.com * start date: Dec 12 17:16:35 2022 GMT * expire date: Dec 12 17:16:35 2023 GMT * issuer: CN=www.codelab-swp.com * SSL certificate verify result: self signed certificate (18), continuing anyway. * allocate connect buffer! * Establish HTTP proxy tunnel to wikipedia.org:443 > CONNECT wikipedia.org:443 HTTP/1.1 > Host: wikipedia.org:443 > User-Agent: curl/7.74.0 > Proxy-Connection: Keep-Alive > * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): < HTTP/1.1 403 Forbidden < content-length: 13 < content-type: text/plain < date: Mon, 12 Dec 2022 19:35:09 GMT < connection: close < * Received HTTP code 403 from proxy after CONNECT * CONNECT phase completed! * Closing connection 0 curl: (56) Received HTTP code 403 from proxy after CONNECT
Feel free to try other domains as well to verify behavior.
Exit the SSH session to "clienta" and initiate a new SSH connection to "clientb."
gcloud compute ssh clientb --zone=$zone --tunnel-through-iap
Run some curl commands to check behavior:
curl https://google.com
This should work as expected clientb VM:
davidtu@clientb:~$ curl https://google.com <HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8"> <TITLE>301 Moved</TITLE></HEAD><BODY> <H1>301 Moved</H1> The document has moved <A HREF="https://www.google.com/">here</A>. </BODY></HTML>
Testing against a org domain:
curl https://wikipedia.org
This works as expected as clientb is not leveraging Cloud SWP:
davidtu@clientb:~$ curl https://wikipedia.org <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>301 Moved Permanently</title> </head><body> <h1>Moved Permanently</h1> <p>The document has moved <a href="https://www.wikipedia.org/">here</a>.</p> </body></html>
Test sending traffic explicitly through Cloud SWP:
curl -x https://10.10.10.50:443/ https://wikipedia.org --proxy-insecure
We see that this traffic is denied via Cloud SWP policy:
davidtu@clientb:~$ curl -x https://10.10.10.50:443/ https://wikipedia.org --proxy-insecure curl: (56) Received HTTP code 403 from proxy after CONNECT
As you have verified, traffic leveraging Cloud SWP is being enforced against the security policy configured. Traffic destined for .com is allowed and all other destinations are denied.
Exit from clientb.
14. Update a Gateway Security Policy Rule for ApplicationMatching
Let's update the rule to match against application level details. We will build a rule to look at the request path and only allow it if it matches index.html.
cat > /tmp/rule-com.yaml << EOF
name: projects/${project_id}/locations/${region}/gatewaySecurityPolicies/${prefix}-policy/rules/rule-com
enabled: true
priority: 1
description: Allow .com traffic with path index.html
basicProfile: ALLOW
sessionMatcher: host().endsWith('com')
applicationMatcher: request.path.matches('index.html')
EOF
Now we can bind the updated rule to the gateway security policy:
gcloud network-security gateway-security-policies rules import rule-com --source=/tmp/rule-com.yaml --location=${region} --gateway-security-policy=${prefix}-policy
15. Testing ApplicationMatcher Rule
SSH into clienta compute VM. This VM has the environmental variable set to use Cloud SWP.
From cloudshell:
gcloud compute ssh clienta --zone=$zone --tunnel-through-iap
Run some web queries to validate functionality. We require –proxy-insecure because we created a self-signed certificate for this lab:
curl http://google.com --proxy-insecure
Notice that this query will fail when it previously passed.
Access denied
Any request path besides "index.html" should get blocked with a 403. Feel free to test this further.
Modify the query to include the path /index.html
curl http://google.com/index.html --proxy-insecure
This request should be successful:
davidtu@clienta:~$ curl http://google.com/index.html --proxy-insecure <HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8"> <TITLE>301 Moved</TITLE></HEAD><BODY> <H1>301 Moved</H1> The document has moved <A HREF="https://www.google.com/index.html">here</A>. </BODY></HTML>
It is expected that we see a 301 redirect since the website is redirecting to http://www.google.com/index.html
Notice that this is an HTTP request. Next, you will need to enable SWP to have TLS inspection capabilities.
Next run the same query but over TLS:
curl -k https://google.com/index.html --proxy-insecure
Expected output:
curl: (56) Received HTTP code 403 from proxy after CONNECT
This request should fail since SWP is not configured to inspect TLS and can not evaluate the path against the applicationMatcher rule.
Exit from clenta.
16. Enable TLS Inspection
Without TLS Inspection, applicationMatcher will not match against HTTPS traffic.
"applicationMatcher" allows filtering on the following:
- Request headers map
- Request method
- Request host
- Request path
- Request query
- Request scheme
- Full request url
- Request useragent
Create Service Account
This service account will have permissions to generate certificates for SWP TLS inspection.
gcloud beta services identity create \
--service=networksecurity.googleapis.com \
--project=$project_id
Ensure CAS is enabled
gcloud services enable privateca.googleapis.com
Create a CA pool
gcloud privateca pools create $prefix-ca-pool \
--tier=devops \
--project=$project_id \
--location=$region
Create Root CA
CA used for certificate signing.
gcloud privateca roots create $prefix-root-ca --pool=$prefix-ca-pool \ --location=$region \ --auto-enable \ --subject="CN=my-swp-ca, O=SWP LLC"
Create a certificate issuance policy file
cat > /tmp/tls-issuance-policy.yaml << EOF
maximumLifetime: 1209600s
baselineValues:
caOptions:
isCa: false
keyUsage:
extendedKeyUsage:
serverAuth: true
EOF
Create TLS Inspection yaml file
cat > /tmp/tls-inspection-policy.yaml << EOF caPool: projects/$project_id/locations/$region/caPools/$prefix-ca-pool name: projects/$project_id/locations/$region/tlsInspectionPolicies/$prefix-tls-inspection EOF
Create TLS Inspection Policy
gcloud network-security tls-inspection-policies import $prefix-tls-inspection \
--source=/tmp/tls-inspection-policy.yaml \
--location=$region
Update the CA Pool to use the certificate issuance policy
gcloud privateca pools update $prefix-ca-pool --issuance-policy=/tmp/tls-issuance-policy.yaml --location=$region
Grant Permissions
This allows your service account to use the CA pool to generate certificates.
gcloud privateca pools add-iam-policy-binding $prefix-ca-pool \
--member=$member \
--role='roles/privateca.certificateManager' \
--location=$region
Update Policy yaml to include TLS inspection
cat > /tmp/policy.yaml << EOF
description: some policy description
name: projects/${project_id}/locations/${region}/gatewaySecurityPolicies/${prefix}-policy
tlsInspectionPolicy: projects/${project_id}/locations/${region}/tlsInspectionPolicies/${prefix}-tls-inspection
EOF
Run the command apply the updated policy
gcloud network-security gateway-security-policies import ${prefix}-policy --source=/tmp/policy.yaml --location=${region}
Update rules to include TLS inspection
Next specify what rules should have TLS inspection "enabtlsInspectionEnabled: true" flag.
cat > /tmp/rule-com.yaml << EOF
name: projects/${project_id}/locations/${region}/gatewaySecurityPolicies/${prefix}-policy/rules/rule-com
enabled: true
priority: 1
description: Allow .com traffic with path index.html
basicProfile: ALLOW
sessionMatcher: host().endsWith('com')
applicationMatcher: request.path.matches('index.html')
tlsInspectionEnabled: true
EOF
Run the command to apply the updated rule
gcloud network-security gateway-security-policies rules import rule-com --source=/tmp/rule-com.yaml --location=${region} --gateway-security-policy=${prefix}-policy
17. Test TLS Inspection
SSH into clienta compute VM. This VM has the environmental variable set to use Cloud SWP.
From cloudshell:
gcloud compute ssh clienta --zone=$zone --tunnel-through-iap
Run the previous web query to verify if SWP is performing TLS inspection to retrieve the path
curl -k https://google.com/index.html --proxy-insecure
This time, it should be successful as SWP can evaluate the ApplicationMatcher.
Expected output:
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8"> <TITLE>301 Moved</TITLE></HEAD><BODY> <H1>301 Moved</H1> The document has moved <A HREF="https://www.google.com/index.html">here</A>. </BODY></HTML>
We've successfully set up Cloud SWP to inspect TLS and evaluate applicationMatcher logic!
Exit from clienta.
18. Cleanup steps
From Cloud Shell, remove the SWP gateway, security policy, certificates, instances, Cloud NAT and Cloud Router:
gcloud -q network-services gateways delete ${prefix}-swp --location=${region}
gcloud -q network-security gateway-security-policies rules delete rule-com --location=${region} --gateway-security-policy=${prefix}-policy
gcloud -q network-security gateway-security-policies delete ${prefix}-policy --location=${region}
gcloud -q certificate-manager certificates delete ${prefix}-cert --location=${region}
gcloud -q network-security tls-inspection-policies delete $prefix-tls-inspection --location=$region
gcloud -q privateca roots disable $prefix-root-ca --pool=$prefix-ca-pool --location=$region
gcloud -q privateca roots delete $prefix-root-ca --pool=$prefix-ca-pool --location=$region --ignore-active-certificates --skip-grace-period
gcloud -q privateca pools delete $prefix-ca-pool --location=$region
gcloud -q compute instances delete clienta --zone=$zone
gcloud -q compute instances delete clientb --zone=$zone
gcloud -q compute routers nats delete ${prefix}-nat-gw-${region} \
--router=$prefix-cr --router-region=$region
gcloud -q compute routers delete `gcloud compute routers list --regions=$region --format="value(NAME)" | grep -e swg-autogen -e codelab-swp` --region=$region
Remove the subnets, FW rules, and VPCs:
gcloud -q compute networks subnets delete $prefix-vpc-subnet \
--region $region
gcloud -q compute networks subnets delete $prefix-proxy-only-subnet \
--region=$region
gcloud -q compute firewall-rules delete $prefix-allow-iap-proxy
gcloud -q compute networks delete $prefix-vpc
19. Congratulations!
Congratulations for completing the codelab. You've successfully configured and deployed Cloud Secure Web Proxy on Google Cloud.
What we've covered
- Cloud SWP and the benefits!