Looker PSC Southbound HTTPS Internet NEG

1. Introduction

NOTE: Looker supports Southbound Producers in any region. For Northbound access to Looker, the consumer endpoint or backend must be in the same region as the Looker PSC instance.

In this codelab you will perform a southbound HTTPS connection to GitHub using an internal tcp proxy load balancer and internet network endpoint group (NEG) invoked from Looker PSC as a Service Consumer.

Private Service Connect is a capability of Google Cloud networking that allows consumers to access managed services privately from inside their VPC network. Similarly, it allows managed service producers to host these services in their own separate VPC networks and offer a private connection to their consumers. For example, when you use Private Service Connect to access Looker, you are the service consumer, and Google is the service producer, as highlighted in Figure 1.

Figure 1.

Southbound access, also known as reverse PSC, enables the Consumer to create a Published Service as a Producer to allow Looker access to endpoints on-premises, in a VPC, to managed services and the Internet. Southbound connections can be deployed in any region, irrespective of where Looker PSC is deployed, as highlighted in Figure 2.

Figure 2.

What you'll learn

• Network requirements
• Create a Private Service Connect producer service
• Create a Private Service Connect endpoint in Looker
• Establish connectivity to GitHub from Looker using a Test Connection

What you'll need

• Google Cloud Project with Owner permissions
• GitHub account and repository
• GitHub Personal Access Token (Classic)
• Existing Looker PSC Instance

2. What you'll build

You'll establish a Producer network, looker-psc-demo, to deploy internal tcp proxy load balancer and Internet NEG published as a service via Private Service Connect (PSC). Once published, you'll perform the following actions to validation access to the Producer service:

• Create a PSC Endpoint in Looker associated with the Producer Service Attachment
• Use the Looker Console to create a new project and test HTTPS connectivity to GitHub.com

3. Network requirements

Below is the breakdown of network requirements for the Producer network, the consumer in this codelab is the Looker PSC instance.

Components	Description
VPC (looker-psc-demo)	Custom mode VPC
PSC NAT Subnet	Packets from the consumer VPC network are translated using source NAT (SNAT) so that their original source IP addresses are converted to source IP addresses from the NAT subnet in the producer's VPC network.
PSC forwarding rule subnet	Used to allocate an IP address for the Regional Internal TCP Proxy Load Balancer
PSC NEG Subnet	Used to allocate an IP address for the Network Endpoint Group
Proxy Only Subnet	Each of the load balancer's proxies is assigned an internal IP address. Packets sent from a proxy to a backend VM or endpoint has a source IP address from the proxy-only subnet.
Internet NEG	A resource used to define an external backend for the load balancer. The endpoint cannot be reachable only over Cloud VPN or Cloud Interconnect.
Backend Service	A backend service acts as a bridge between your load balancer and your backend resources. In the tutorial, the backend service is associated with the Internet NEG.
Cloud Router	Cloud NAT relies on Cloud Routers for control plane capabilities, but not for BGP session management.
Cloud NAT	The regional internet NEG leverages Cloud NAT for internet egress.

4. Codelab topology

5. Setup and Requirements

Self-paced environment setup

1. Sign-in to the Google Cloud Console and create a new project or reuse an existing one. If you don't already have a Gmail or Google Workspace account, you must create one.

• The Project name is the display name for this project's participants. It is a character string not used by Google APIs. You can always update it.
• The Project ID is unique across all Google Cloud projects and is immutable (cannot be changed after it has been set). The Cloud Console auto-generates a unique string; usually you don't care what it is. In most codelabs, you'll need to reference your Project ID (typically identified as PROJECT_ID). If you don't like the generated ID, you might generate another random one. Alternatively, you can try your own, and see if it's available. It can't be changed after this step and remains for the duration of the project.
• For your information, there is a third value, a Project Number, which some APIs use. Learn more about all three of these values in the documentation.

Caution: A project ID is globally unique and can't be used by anyone else after you've selected it. You are the only user of that ID. Even if a project is deleted, the ID can't be used again

Note: If you use a Gmail account, you can leave the default location set to No organization. If you use a Google Workspace account, choose a location that makes sense for your organization.

2. Next, you'll need to enable billing in the Cloud Console to use Cloud resources/APIs. Running through this codelab won't cost much, if anything at all. To shut down resources to avoid incurring billing beyond this tutorial, you can delete the resources you created or delete the project. New Google Cloud users are eligible for the $300 USD Free Trial program.

Start Cloud Shell

While Google Cloud can be operated remotely from your laptop, in this codelab you will be using Google Cloud Shell, a command line environment running in the Cloud.

From the Google Cloud Console, click the Cloud Shell icon on the top right toolbar:

It should only take a few moments to provision and connect to the environment. When it is finished, you should see something like this:

This virtual machine is loaded with all the development tools you'll need. It offers a persistent 5GB home directory, and runs on Google Cloud, greatly enhancing network performance and authentication. All of your work in this codelab can be done within a browser. You do not need to install anything.

6. Before you begin

Enable APIs

Inside Cloud Shell, make sure that your project id is set up:

gcloud config list project
gcloud config set project [YOUR-PROJECT-ID]
project=[YOUR-PROJECT-ID]
region=[YOUR-REGION]
echo $project
echo $region

Enable all necessary services:

gcloud services enable compute.googleapis.com

7. Create Producer VPC Network

VPC Network

Inside Cloud Shell, perform the following:

gcloud compute networks create looker-psc-demo --subnet-mode custom

Create Subnets

The PSC subnet will be associated with the PSC Service Attachment for the purpose of Network Address Translation.

Inside Cloud Shell, create the PSC NAT Subnet:

gcloud compute networks subnets create producer-psc-nat-subnet --network looker-psc-demo --range 172.16.10.0/28 --region $region --purpose=PRIVATE_SERVICE_CONNECT

Inside Cloud Shell, create the producer forwarding rule subnet:

gcloud compute networks subnets create producer-psc-fr-subnet --network looker-psc-demo --range 172.16.20.0/28 --region $region --enable-private-ip-google-access

Inside Cloud Shell, create the producer regional proxy only subnet:

gcloud compute networks subnets create $region-proxy-only-subnet \
  --purpose=REGIONAL_MANAGED_PROXY \
  --role=ACTIVE \
  --region=$region \
  --network=looker-psc-demo \
  --range=10.10.10.0/24

Create the Public NAT gateway

The NAT Gateway is used by the regional internal tcp proxy load balancer for internet egress with the configuration option, –endpoint-types=ENDPOINT_TYPE_MANAGED_PROXY_LB, therefore the same NATGW will not support GCE/GKE internet egress. Deploy an additional NAT GW with the –endpoint-types=ENDPOINT_TYPE_VM for GCE/GKE internet egress.

Inside Cloud Shell, create the Cloud Router:

gcloud compute routers create looker-psc-demo-cloud-router --network looker-psc-demo --region $region

Inside Cloud Shell, create the Cloud NAT gateway enabling internet egress for the tcp proxy load balancer:

gcloud compute routers nats create looker-psc-demo-natgw \
  --router=looker-psc-demo-cloud-router \
  --endpoint-types=ENDPOINT_TYPE_MANAGED_PROXY_LB \
  --nat-custom-subnet-ip-ranges=$region-proxy-only-subnet \
  --auto-allocate-nat-external-ips \
  --region=$region

Reserve the load balancer's IP address

Inside Cloud Shell, reserve an internal IP address for the load balancer:

gcloud compute addresses create internet-neg-lb-ip \
  --region=$region \
  --subnet=producer-psc-fr-subnet

Inside Cloud Shell, view the reserved IP Address.

gcloud compute addresses describe internet-neg-lb-ip \
  --region=$region | grep -i address:

Example output:

user@cloudshell$ gcloud compute addresses describe internet-neg-lb-ip   --region=$region | grep -i address:
address: 172.16.20.2

Set up the Internet NEG

Create an Internet NEG, and set the –network-endpoint-type to internet-fqdn-port (the hostname and port where your external backend can be reached).

Inside Cloud Shell, create a Internet NEG used for github.com

gcloud compute network-endpoint-groups create github-internet-neg \
    --network-endpoint-type=INTERNET_FQDN_PORT \
    --network=looker-psc-demo \
    --region=$region

NOTE: fqdn=github.com,port=443 is a required to support HTTPS

Inside Cloud Shell, update the Internet NEG github-internet-neg with the FQDN github.com and port 443

gcloud compute network-endpoint-groups update github-internet-neg \
    --add-endpoint="fqdn=github.com,port=443" \
    --region=$region

Create Network Firewall Policy and Firewall Rules

Inside Cloud Shell, perform the following:

gcloud compute network-firewall-policies create looker-psc-demo-policy --global

gcloud compute network-firewall-policies associations create --firewall-policy looker-psc-demo-policy --network looker-psc-demo --name looker-psc-demo --global-firewall-policy

The following firewall rule allows traffic from the PSC NAT Subnet range to all instances in the network.

Inside Cloud Shell, perform the following:

gcloud compute network-firewall-policies rules create 2001 --action ALLOW --firewall-policy looker-psc-demo-policy --description "allow traffic from PSC NAT subnet" --direction INGRESS --src-ip-ranges 172.16.10.0/28 --global-firewall-policy --layer4-configs=tcp

8. Create Producer Service

Create Load Balancer Components

Inside Cloud Shell, perform the following:

gcloud compute backend-services create producer-backend-svc  --protocol=tcp --region=$region --load-balancing-scheme=INTERNAL_MANAGED

gcloud compute backend-services add-backend producer-backend-svc --network-endpoint-group=github-internet-neg --network-endpoint-group-region=$region --region=$region

In Cloud Shell, Create a target TCP proxy to route requests to your backend service:

gcloud compute target-tcp-proxies create producer-lb-tcp-proxy \
      --backend-service=producer-backend-svc  \
      --region=$region

In the following syntax, create a forwarding rule (internal tcp proxy load balancer).

In Cloud Shell, perform the following:

gcloud compute forwarding-rules create producer-github-fr \
     --load-balancing-scheme=INTERNAL_MANAGED \
     --network-tier=PREMIUM \
     --network=looker-psc-demo \
     --subnet=producer-psc-fr-subnet \
     --address=internet-neg-lb-ip \
     --target-tcp-proxy=producer-lb-tcp-proxy \
     --target-tcp-proxy-region=$region \
     --region=$region \
     --ports=443

Create Service Attachment

Inside Cloud Shell, create the Service Attachment, github-svc-attachment-https:

gcloud compute service-attachments create github-svc-attachment-https --region=$region --producer-forwarding-rule=producer-github-fr --connection-preference=ACCEPT_AUTOMATIC --nat-subnets=producer-psc-nat-subnet

Next, obtain and note the Service Attachment listed in the selfLink URI starting with projects to configure the PSC endpoint in Looker.

selfLink: projects/<your-project-id>/regions/<your-region>/serviceAttachments/github-svc-attachment-https

Inside Cloud Shell, perform the following:

gcloud compute service-attachments describe github-svc-attachment-https --region=$region

Example:

connectionPreference: ACCEPT_AUTOMATIC
creationTimestamp: '2024-08-30T09:44:03.883-07:00'
description: ''
enableProxyProtocol: false
fingerprint: RfKh3blWZE0=
id: '2897904404386302012'
kind: compute#serviceAttachment
name: github-svc-attachment-https
natSubnets:
- https://www.googleapis.com/compute/v1/projects/$project/regions/$region/subnetworks/producer-psc-nat-subnet
pscServiceAttachmentId:
  high: '19348441121424360'
  low: '2897904404386302012'
reconcileConnections: false
region: https://www.googleapis.com/compute/v1/projects/$project/regions/$region
selfLink: https://www.googleapis.com/compute/v1/projects/$project/regions/$region/serviceAttachments/github-svc-attachment-https
targetService: https://www.googleapis.com/compute/v1/projects/$project/regions/$region/forwardingRules/producer-github-fr

In Cloud Console, navigate to:

Network Services → Private Service Connect → Published Services

9. Establish a PSC Endpoint Connection in Looker

In the following section, you will associate the Producers Service Attachment with Looker Core PSC through the use –psc-service-attachment flags in Cloud Shell for a single domain.

NOTE: Each time you run an update command with –psc-service-attachment flags, you must include every connection that you want to be enabled, including connections that were already enabled previously. Reference Looker PSC docs for an example.

NOTE: The flag domain=github.com is required to establish a HTTPS connection to github.com

NOTE: You're Looker PSC Instance needs to be enabled before proceeding.

Inside Cloud Shell, create the psc association by updating the following parameters to match your environment:

• INSTANCE_NAME: The name of your Looker (Google Cloud core) instance.
• DOMAIN_1: github.com
• SERVICE_ATTACHMENT_1: URI captured when describing the Service Attachment, github-svc-attachment-https.
• REGION: The region in which your Looker (Google Cloud core) instance is hosted.

Inside Cloud Shell, perform the following:

gcloud looker instances update INSTANCE_NAME \
--psc-service-attachment  domain=DOMAIN_1,attachment=SERVICE_ATTACHMENT_URI_1 \
--region=REGION

Example:

gcloud looker instances update looker-psc-instance \
--psc-service-attachment  domain=github.com,attachment=projects/$project/regions/$region/serviceAttachments/github-svc-attachment-https \
--region=$region

Inside Cloud Shell, validate the serviceAttachments connectionStatus is "ACCEPTED", update with your Looker PSC INSTANCE_NAME

gcloud looker instances describe [INSTANCE_NAME] --region=$region --format=json

Example:

gcloud looker instances describe looker-psc-instance --region=$region --format=json

Example:

{
  "adminSettings": {},
  "createTime": "2024-08-23T00:00:45.339063195Z",
  "customDomain": {
    "domain": "cosmopup.com",
    "state": "AVAILABLE"
  },
  "encryptionConfig": {},
  "lookerVersion": "24.12.28",
  "name": "projects/$project/locations/$region/instances/looker-psc-instance",
  "platformEdition": "LOOKER_CORE_ENTERPRISE_ANNUAL",
  "pscConfig": {
    "allowedVpcs": [

    "projects/$project/global/networks/looker-psc-demo",
      "projects/$project/global/networks/looker-shared-vpc"
    ],
    "lookerServiceAttachmentUri": "projects/t7ec792caf2a609d1-tp/regions/$region/serviceAttachments/looker-psc-f51982e2-ac0d-48b1-91bb-88656971c183",
    "serviceAttachments": [
      {
        "connectionStatus": "ACCEPTED",
        "localFqdn": "github.com",
        "targetServiceAttachmentUri": "projects/$project/regions/$region/serviceAttachments/github-svc-attachment-https"
      }
    ]
  },
  "pscEnabled": true,
  "state": "ACTIVE",
  "updateTime": "2024-08-30T17:47:33.440271635Z"
}

Validate the PSC endpoint in Cloud Console

From Cloud Console you can validate the PSC Connection

In Cloud Console, navigate to:

Looker → Looker Instance → Details

10. Test Connectivity to GitHub

In the following steps, you'll use Looker Console to create a project to validate HTTPS connectivity to github.com.

11. Create a new project

Enable Development mode

In Looker Console, navigate to:

Enable Development Mode (bottom left page), once selected the banner ‘You are in Development Mode' is displayed.

Create a new project

In Cloud Console, navigate to:

Develop → Projects

Select New LookML Project

Provide a project name, select Blank Project then Create Project.

Select Configure Git

Configure Git

Update the Repository URL with your HTTPS github details, ensure to append the URL with .git then select Continue.

Example:

Update the selection with your GitHub username and Personal Access token (classic), then select Test and Finalize Setup.

Select Git Actions

Select Test Git Connection

Validate the Git Connection Test

Clean up

From a single Cloud Shell terminal delete lab components

gcloud compute service-attachments delete github-svc-attachment-https --region=$region -q

gcloud compute forwarding-rules delete producer-github-fr --region=$region -q

gcloud compute target-tcp-proxies delete producer-lb-tcp-proxy --region=$region -q

gcloud compute backend-services delete producer-backend-svc --region=$region -q

gcloud compute network-firewall-policies rules delete 2001 --firewall-policy looker-psc-demo-policy --global-firewall-policy -q

gcloud compute network-firewall-policies associations delete --firewall-policy=looker-psc-demo-policy  --name=looker-psc-demo --global-firewall-policy -q

gcloud compute network-firewall-policies delete looker-psc-demo-policy --global -q

gcloud compute routers nats delete looker-psc-demo-natgw --router=looker-psc-demo-cloud-router --router-region=$region -q

gcloud compute routers delete looker-psc-demo-cloud-router --region=$region -q

gcloud compute network-endpoint-groups delete github-internet-neg --region=$region -q

gcloud compute addresses delete internet-neg-lb-ip --region=$region -q

gcloud compute networks subnets delete producer-psc-fr-subnet producer-psc-nat-subnet $region-proxy-only-subnet --region=$region -q

gcloud compute networks delete looker-psc-demo -q

12. Congratulations

Congratulations, you've successfully configured and validated connectivity to GitHub using Looker Console powered by Private Service Connect.

You created the producer infrastructure, learned how to create an Internet NEG, Producer Service and Looker PSC endpoint that allowed connectivity to the Producer service.

Cosmopup thinks codelabs are awesome!!

What's next?

Check out some of these codelabs...

• Using Private Service Connect to publish and consume services
• Connect to on-prem services over Hybrid Networking using Private Service Connect and an internal TCP Proxy load balancer
• Access to all published Private Service Connect codelabs

Further reading & Videos

• Setting up and testing a Git connection | Looker | Google Cloud
• Private Service Connect overview

Reference docs

• Set up a regional internal proxy Network Load Balancer with an external backend
• Cloud NAT overview
• Create a Looker (Google Cloud core) Private Service Connect instance
• How to publish a service using Private Service Connect