About this codelab
1. Overview
There are several options to connect to a private Cloud SQL instance from another project or VPC network. You can use Private services access (PSA) to reach the internal IP addresses of Google and third-party services through private connections. To extend access to the database you can now also enable Private Service Connect (PSC).
Private Service Connect (PSC) allows service producers to expose their services via service attachments, which consumers can use to create PSC endpoints and/or PSC backends in their environment. Through these PSC endpoints they can connect to the producer services via a user specified private IP.
In this lab you can configure and test the option.
In this lab, you're going to build a simple architecture that illustrates the use of PSA & PSC endpoint access with CloudSQL.
Figure1.
For this lab, you will need two projects or separate VPCs in the same project.
Objectives
In this lab you will learn how to perform the following task:
- Enable service networking
- Set up PSA
- Create a CloudSQL instance with PSA
- Enable PSC service attachment.
- Create a PSC endpoint in the consumer VPC to connect to the CloudSQL database
- Verify access to the SQL database from test VMs, in both producer and consumer VPCs
2. Setup and Requirements
Self-paced environment setup
- Sign-in to the Google Cloud Console and create a new project or reuse an existing one. If you don't already have a Gmail or Google Workspace account, you must create one.
- The Project name is the display name for this project's participants. It is a character string not used by Google APIs. You can always update it.
- The Project ID is unique across all Google Cloud projects and is immutable (cannot be changed after it has been set). The Cloud Console auto-generates a unique string; usually you don't care what it is. In most codelabs, you'll need to reference your Project ID (typically identified as
PROJECT_ID
). If you don't like the generated ID, you might generate another random one. Alternatively, you can try your own, and see if it's available. It can't be changed after this step and remains for the duration of the project. - For your information, there is a third value, a Project Number, which some APIs use. Learn more about all three of these values in the documentation.
- Next, you'll need to enable billing in the Cloud Console to use Cloud resources/APIs. Running through this codelab won't cost much, if anything at all. To shut down resources to avoid incurring billing beyond this tutorial, you can delete the resources you created or delete the project. New Google Cloud users are eligible for the $300 USD Free Trial program.
Start Cloud Shell
While Google Cloud can be operated remotely from your laptop, in this codelab you will be using Google Cloud Shell, a command line environment running in the Cloud.
From the Google Cloud Console, click the Cloud Shell icon on the top right toolbar:
It should only take a few moments to provision and connect to the environment. When it is finished, you should see something like this:
This virtual machine is loaded with all the development tools you'll need. It offers a persistent 5GB home directory, and runs on Google Cloud, greatly enhancing network performance and authentication. All of your work in this codelab can be done within a browser. You do not need to install anything.
3. Task 1. Setup environment for database project with (Terraform)
In the database project, we will enable service networking, create a VPC, create a PSA range, create a service networking peering, and create firewall rules. Open the cloud console and select the project you will be using for the database.
- Open Cloud Shell located at the top of your console on the right, ensure you see the database Project ID for the database project in Cloud Shell, confirm any prompts to allow access.
- Create a folder called terraform-db-psc and move to folder
mkdir terraform-db-psc && cd terraform-db-psc
- Create a main.tf, variable.tf, nat-vm.tf and database.tf file.
touch main.tf variable.tf nat-vm.tf database.tf
- Switch over to the Cloud Shell editor view. Select editor, ensure you allow any necessary prompts so the interface can load.
- Once loaded navigate to, File > Open Folder and go to the /home/your-user-name/terraform-db-psc and select Ok to open the folder in the editor.
- Select the variable.tf file and add the following. Replace the
your-database-project-id-here
text with your actual database project ID in quotes. Replace theyour-consumer-project-id-here
text with your actual consumer project ID in quotes.
variable "project_id" { type = string default = "your-database-project-id-here" } variable "project_id_consumer" { type = set(string) default = ["your-consumer-project-id-here"] } variable "network_id" { type = string default = "database-net" } variable "region_id" { type = string default = "us-east1" } variable "db_password" { type = string default = "cloudsql24" }
- Next, open the main.tf file. We are going to add some terraform code to perform various actions as explained below.
Enable API's |
|
Create VPC |
|
Add Firewall rules |
|
Create PSA range |
|
Create a peering |
|
- Copy and paste the following into the main .tf file.
resource "google_project_service" "default" { for_each = toset([ "compute.googleapis.com", "servicenetworking.googleapis.com", "sqladmin.googleapis.com", "cloudresourcemanager.googleapis.com" ]) service = each.value disable_on_destroy = false } resource "google_compute_network" "default" { project = var.project_id name = var.network_id auto_create_subnetworks = true mtu = 1460 routing_mode = "GLOBAL" } resource "google_compute_global_address" "psa_range" { name = "psa-range-${google_compute_network.default.name}" project = var.project_id purpose = "VPC_PEERING" address_type = "INTERNAL" prefix_length = 20 address = "172.18.0.0" network = google_compute_network.default.id } resource "google_service_networking_connection" "private_connection" { network = google_compute_network.default.self_link service = "servicenetworking.googleapis.com" reserved_peering_ranges = [ google_compute_global_address.psa_range.name, ] } resource "google_compute_firewall" "allow_icmp" { name = "allow-icmp-${google_compute_network.default.name}" network = google_compute_network.default.id project = var.project_id allow { protocol = "icmp" } source_ranges = ["0.0.0.0/0"] target_tags = ["allow-icmp"] } resource "google_compute_firewall" "allow_ssh" { name = "allow-ssh-${google_compute_network.default.name}" network = google_compute_network.default.id project = var.project_id allow { protocol = "tcp" ports = ["22"] } source_ranges = ["0.0.0.0/0"] target_tags = ["allow-ssh"] } resource "google_compute_firewall" "allow_sql" { name = "allow-sql-${google_compute_network.default.name}" network = google_compute_network.default.id project = var.project_id allow { protocol = "tcp" ports = ["3306"] } source_ranges = ["0.0.0.0/0"] } resource "google_compute_firewall" "allow_internal" { name = "allow-internal-${google_compute_network.default.name}" network = google_compute_network.default.id project = var.project_id allow { protocol = "icmp" } allow { protocol = "tcp" ports = ["0-65535"] } allow { protocol = "udp" ports = ["0-65535"] } source_ranges = ["10.128.0.0/9", "172.18.0.0/20"] # Include PSA range }
- Switch back to Cloud Shell terminal make sure your are in the terraform-db-psc directory
cd terraform-db-psc
and run the following commands
terraform init
Initializes the working directory. This step downloads the providers required for the given configuration.
terraform plan
Generates an execution plan, showing what actions Terraform will take to deploy your infrastructure.
- Now to create the resources run the
terraform apply
command and typeyes
to run.
4. Task 2. Create CloudSQL instance with PSA (Terraform)
Let's use Terraform to create a CloudSQL instance, with the PSA option to allow connectivity. Complete this configuration in the database project.
- Switch over to the Cloud Shell editor view. Ensure you are in the
terraform-db-psc
folder. Select thedatabase.tf
file and add the following.
resource "google_sql_database_instance" "default" {
name = "psc-psa-sql-db"
region = var.region_id
database_version = "MYSQL_8_0"
settings {
tier = "db-f1-micro"
availability_type = "REGIONAL"
backup_configuration {
enabled = true
binary_log_enabled = true
}
ip_configuration {
ipv4_enabled = "false"
private_network = google_compute_network.default.id
}
}
deletion_protection = false
}
resource "google_sql_user" "users" {
name = "testsql"
instance = google_sql_database_instance.default.name
password = var.db_password
}
- Switch back to Cloud Shell terminal, make sure you are in the terraform-db-psc directory. Now to create the resources run the
terraform apply
command and typeyes
to run. Creating the CloudSQL instance with PSA may take up to 15+ minutes.
Verify database
- After completion, Go to SQL in the console, select the newly created psc-psa-sql-db database instance.
- On the left side expand the SQL options slideout and select Connections
- Under the summary tab you can see information about the connection.
- Copy the Internal IP Address and save it somewhere on your system. As you can see it was assigned from the range we reserved.
- You will need this IP to complete task 3 step 5.
- On the left side expand the SQL options slideout and select Users.
- You should see a user called testsql with password cloudsql24 (This will be used to access db later on)
5. Task 3. Create test VM and connect to SQL instance from within db project (Terraform)
Let's use Terraform to create a test VM and NAT gateway. Complete this configuration in the database project.
- Switch over to the Cloud Shell editor view. Ensure you are in the
terraform-db-psc
folder. Select thenat-vm.tf
file and add the following.
resource "google_compute_router" "default" {
name = "outbound-nat"
region = var.region_id
network = google_compute_network.default.id
bgp {
asn = 64514
}
}
resource "google_compute_router_nat" "default" {
name = "outbound-nat-gw"
router = google_compute_router.default.name
region = google_compute_router.default.region
nat_ip_allocate_option = "AUTO_ONLY"
source_subnetwork_ip_ranges_to_nat = "ALL_SUBNETWORKS_ALL_IP_RANGES"
log_config {
enable = true
filter = "ERRORS_ONLY"
}
}
resource "google_compute_instance" "sql_net_vm" {
name = "db-vm-test"
machine_type = "e2-medium"
zone = "${var.region_id}-b"
project = var.project_id
boot_disk {
initialize_params {
image = "debian-cloud/debian-11"
}
}
network_interface {
network = google_compute_network.default.self_link
}
# No external IP address
network_performance_config {
total_egress_bandwidth_tier = "DEFAULT"
}
tags = ["allow-icmp", "allow-ssh", "allow-internal", "allow-health-check"]
# Install MariaDB at startup
metadata_startup_script = <<EOF
#!/bin/bash
apt-get update
apt-get install -y mariadb-server
systemctl enable mariadb
EOF
}
- Switch back to Cloud Shell terminal, make sure you are in the terraform-db-psc directory. Now to create the resources run the
terraform plan, then terraform apply
command and typeyes
to run. This will create a NAT gateway and VM with MariaDB installed.
Verify database
Complete this in the consumer project
- Go to VM Instance. Select db-vm-test
- Select the SSH option to connect to the VM.
- Connect to DB via the local IP you copied before with the following:
mysql --host=172.18.0.2 --user=testsql --password=cloudsql24
p.s Change the IP address to your actual IP for it to connect.
- When connection is established let create a database with the following localvpcsqldb.
CREATE DATABASE localvpcsqldb;
- Once complete type exit to exit the connection. Type exit again to close the VM.
6. Task 4. Enable PSC on the CloudSQL instance
Let's use gcloud commands to enable the PSC option to allow connectivity. Complete this configuration in the database project.
- Switch over to the Cloud Shell view.
- Run the following command in Cloud Shell.
- Replace Ensure
YOUR_DB_PROJECT_HERE
with the database project id. - Replace Ensure
YOUR_CONSUMER_PROJECT_HERE
with the consumer project id.
project_id=YOUR_DB_PROJECT_HERE
allowed_project=YOUR_CONSUMER_PROJECT_HERE
gcloud beta sql instances patch psc-psa-sql-db \
--project=$project_id \
--enable-private-service-connect \
--allowed-psc-projects=$allowed_project
- This will update the existing database with PSC attachment. This will take a while 10+ minutes.
If you get a timeout waiting error it should also give you the command to continue to see the progress. Something like "You can continue waiting for the operation by running `gcloud beta sql operations wait –project "
Please note you can deploy a new database with PSA and PSC enabled from inception.
Verify database
- After completion, Go to SQL in the console, select the psc-psa-sql-db database instance.
- On the left side expand the SQL options slideout and select Connections
- Under the summary tab you can see information about the connection.
- Copy the Service attachment address and save it somewhere on your system.
- You will need this to complete the next task 5 step 8.
7. Task 5. Setup environment for consumer project (Terraform)
In the consumer project, we will create a custom VPC with Firewall rules and subnet. Open the cloud console and select the project you will be using.
- At the top of your console on the right next to Google Cloud icon, click the dropdown and then select the all tab.
- From the listed projects select your consumer project ID to continue.
- Open Cloud Shell located at the top of your console on the right, ensure you see the consumer project ID in Cloud Shell, confirm any prompts to allow access.
- Create a folder called terraform-consumer-psc and move to folder
mkdir terraform-consumer-psc && cd terraform-consumer-psc
- Create a main.tf, variable.tf and psc.tf file.
touch main.tf variable.tf psc.tf
- Switch over to the Cloud Shell editor view for the consumer project ID. Select editor, ensure you allow any necessary prompts so the interface can load.
- Once loaded navigate to, File > Open Folder and go to the /home/your-user-name/terraform-consumer-psc and select Ok to open the folder in the editor.
- Select the variable.tf file and add the following. Replace the
your-consumer-project-id-here
text with your actual consumer project ID in quotes Replace theyour-psc-attachment-id-here
text with your psc attachment address in quotes. This is the ID you copied in task 4 section 5-7
variable "project_id" { type = string default = "your-consumer-project-id-here" } variable "psc_attachment_id" { type = string default = "your-psc-attachment-id-here" } variable "network_id" { type = string default = "psc-endpoint-vpc" } variable "region_id" { type = string default = "us-east1" }
- Next, open the main.tf file. We are going to add some terraform code to perform various actions as explained below.
Enable API's |
|
Create VPC |
|
Create NAT gateway |
|
Add Firewall rules |
|
- Copy and paste the following into the main .tf file.
resource "google_project_service" "default" { for_each = toset([ "compute.googleapis.com", "cloudresourcemanager.googleapis.com" ]) service = each.value disable_on_destroy = false } resource "google_compute_network" "default" { project = var.project_id name = var.network_id auto_create_subnetworks = false mtu = 1460 routing_mode = "GLOBAL" } resource "google_compute_subnetwork" "vm_subnet_pc" { name = "vm-net" ip_cidr_range = "192.168.80.0/24" region = var.region_id network = google_compute_network.default.name project = var.project_id } resource "google_compute_subnetwork" "psc_subnet_ip" { name = "psc-subnet" ip_cidr_range = "192.168.100.0/24" region = var.region_id network = google_compute_network.default.name project = var.project_id } resource "google_compute_router" "default" { name = "outbound-nat" region = var.region_id network = google_compute_network.default.id bgp { asn = 64514 } } resource "google_compute_router_nat" "default" { name = "outbound-nat-gw" router = google_compute_router.default.name region = google_compute_router.default.region nat_ip_allocate_option = "AUTO_ONLY" source_subnetwork_ip_ranges_to_nat = "ALL_SUBNETWORKS_ALL_IP_RANGES" log_config { enable = true filter = "ERRORS_ONLY" } } resource "google_compute_firewall" "allow_icmp" { name = "allow-icmp-${google_compute_network.default.name}" network = google_compute_network.default.id project = var.project_id allow { protocol = "icmp" } source_ranges = ["0.0.0.0/0"] } resource "google_compute_firewall" "allow_ssh" { name = "allow-ssh-${google_compute_network.default.name}" network = google_compute_network.default.id project = var.project_id allow { protocol = "tcp" ports = ["22"] } source_ranges = ["0.0.0.0/0"] target_tags = ["allow-ssh"] } resource "google_compute_firewall" "allow_internal" { name = "allow-internal-${google_compute_network.default.name}" network = google_compute_network.default.id project = var.project_id allow { protocol = "icmp" } allow { protocol = "tcp" ports = ["0-65535"] } allow { protocol = "udp" ports = ["0-65535"] } source_ranges = ["192.168.0.0/16"] }
- Switch back to Cloud Shell terminal make sure your are in the terraform-consumer-psc directory
cd terraform-consumer-psc
in the consumer project and run the following commands
terraform init
Initializes the working directory. This step downloads the providers required for the given configuration.
terraform plan
Generates an execution plan, showing what actions Terraform will take to deploy your infrastructure.
- Now to create the resources run the
terraform apply
command and typeyes
to run.
8. Task 6. Create the PSC endpoint to SQL DB attachment in consumer project and test VM (Terraform)
Complete this configuration in the consumer project.
- Switch back to Cloud Shell editor, open the terraform-consumer-psc directory and select the
psc.tf
file. Add the following Terraform code to the file. This creates a static internal IP, a PSC endpoint and a test VM with Maria DB installed.
resource "google_compute_address" "default" { name = "psc-ip-1" region = var.region_id address_type = "INTERNAL" subnetwork = google_compute_subnetwork.psc_subnet_ip.name address = "192.168.100.100" } resource "google_compute_forwarding_rule" "default" { name = "psc-sql-endpoint" region = var.region_id network = google_compute_network.default.name ip_address = google_compute_address.default.self_link load_balancing_scheme = "" target = var.psc_attachment_id } resource "google_compute_instance" "sql_test_vm" { name = "remote-sql-test-vm" machine_type = "e2-medium" zone = "${var.region_id}-b" project = var.project_id boot_disk { initialize_params { image = "debian-cloud/debian-11" } } network_interface { subnetwork = google_compute_subnetwork.vm_subnet_pc.name } # No external IP address network_performance_config { total_egress_bandwidth_tier = "DEFAULT" } tags = ["allow-icmp", "allow-ssh", "allow-internal", "allow-health-check"] # Install MariaDB at startup metadata_startup_script = <<EOF #!/bin/bash apt-get update apt-get install -y mariadb-server systemctl enable mariadb EOF }
- Switch back to Cloud Shell terminal make sure your are in the terraform-consumer-psc directory
cd terraform-consumer-psc
in the consumer project and run the following commands
terraform plan
Generates an execution plan, showing what actions Terraform will take to deploy your infrastructure.
- Now to create the resources run the
terraform apply
command and typeyes
to run
9. Task 7. Verify endpoint connectivity from VM via PSC IP address
Complete this in the consumer project
- Go to VM Instance. Select remote-sql-test-vm
- Select the SSH option to connect to the VM.
- Connect to DB via endpoint with the following:
mysql --host=192.168.100.100 --user=testsql --password=cloudsql24
- When connection is established let create a database with the following psc-sql-db.
CREATE DATABASE remotepscpsasqldb;
- Once complete type exit to exit the connection. Type exit again to close the VM.
10. Task 8. Verify database entry
Complete this verification in the database project
- At the top of your console on the right next to Google Cloud icon, click the dropdown and then select the all tab.
- From the listed projects select your database project ID to continue.
- Navigate to SQL, select the database psc-psa-sql-db, expand the SQL options and select Databases. You should see the database you just created called remotepscpsasqldb from the consumer project VM and also the localvpcsqldb from the db project VM**
.
**
11. Clean Up
Complete this in the Consumer project
- Select the consumer project ID
- Go to Cloud Shell make sure you see the consumer project id
- Go to the terraform-consumer-psc directory
cd terraform-consumer-psc
and run the following commandterraform destroy
and typeyes
all the resources you created in your consumer project with Terraform will be removed.
Complete this in the Database project
- Select the database project ID
- Go to Cloud Shell make sure you see the database project id
- Go to the terraform-db-psc directory
cd terraform-db-psc
and run the following commandterraform destroy
and typeyes
most of the resources you created in your db project with Terraform will be removed. The service networking may not be deleted and give an error. - Next delete the following files from the terraform-db-psc folder.
rm -r terraform.tfstate terraform.tfstate.backup
- Next in the console go to VPC networks
- Select database-net VPC, select PRIVATE SERVICES ACCESS, under the Allocated IP Ranges for Services, select the psa-range-database-net and select the Release option
- Next select VPC NETWORK PEERING, select the servicenetworking-googleapis-com and select the Delete option
- Next, select Delete VPC network
12. Congratulations
Congratulations, you've successfully configured a Cloud SQL instance with PSA and PSC attachment and created a Private Service Connect endpoint in another VPC.
Learn more
You can read more about Private Service Connect and Cloud SQL
Other related codelab: https://codelabs.developers.google.com/codelabs/cloudsql-psc-terraform