Creating AlloyDB database with Cloud Run Job

1. Overview

This feature is covered by the Pre-GA Offerings Terms of the Google Cloud Terms of Service. Pre-GA features might have limited support, and changes to pre-GA features might not be compatible with other pre-GA versions. For more information, see the launch stage descriptions.

In this lab, you will create an AlloyDB database with Cloud Run Job. You will configure Private services access and Serverless VPC access to enable connectivity between Cloud Run Job and AlloyDB Database using Private IP.

What you will learn

In this lab, you will learn how to do the following:

• Setup AlloyDB cluster and instance
• Deploy Cloud Run Job to create AlloyDB database

2. Setup and Requirements

Cloud Project setup

1. Sign-in to the Google Cloud Console and create a new project or reuse an existing one. If you don't already have a Gmail or Google Workspace account, you must create one.

• The Project name is the display name for this project's participants. It is a character string not used by Google APIs. You can update it at any time.
• The Project ID must be unique across all Google Cloud projects and is immutable (cannot be changed after it has been set). The Cloud Console auto-generates a unique string; usually you don't care what it is. In most codelabs, you'll need to reference the Project ID (it is typically identified as PROJECT_ID). If you don't like the generated ID, you may generate another random one. Alternatively, you can try your own and see if it's available. It cannot be changed after this step and will remain for the duration of the project.
• For your information, there is a third value, a Project Number which some APIs use. Learn more about all three of these values in the documentation.

Caution: A project ID must be globally unique and cannot be used by anyone else after you've selected it. You are the only user of that ID. Even if a project is deleted, the ID can never be used again

Note: If you're using a Gmail account, you can leave the default location set to No organization. If you're using a Google Workspace account, then choose a location that makes sense for your organization.

2. Next, you'll need to enable billing in the Cloud Console to use Cloud resources/APIs. Running through this codelab shouldn't cost much, if anything at all. To shut down resources so you don't incur billing beyond this tutorial, you can delete the resources you created or delete the whole project. New users of Google Cloud are eligible for the $300 USD Free Trial program.

Environment Setup

Activate Cloud Shell by clicking on the icon to the right of the search bar.

To enable service APIs, copy and paste command below into the terminal and hit Enter:

gcloud services enable \
    alloydb.googleapis.com \
    artifactregistry.googleapis.com \
    cloudbuild.googleapis.com \
    run.googleapis.com \
    vpcaccess.googleapis.com \
    compute.googleapis.com \
    servicenetworking.googleapis.com \
    --quiet

Set environment variables:

export PROJECT_ID=$(gcloud config get-value project)
export PROJECT_NUMBER=$(gcloud projects describe $PROJECT_ID --format='value(projectNumber)')
export PROJECT_NAME=$(gcloud projects describe $PROJECT_ID --format='value(name)')
export DB_DATABASE=test
export DB_USER=postgres
export DB_PASSWORD=password123
export REGION=us-central1
export CLUSTER=test-cluster
export INSTANCE=test-instance
export VPC_CONNECTOR=vpcconnector

3. Setup and Create AlloyDB cluster

Set required permissions for the setup. This will allow Cloud Run Job to connect to AlloyDB database.

gcloud projects add-iam-policy-binding $PROJECT_NAME \
  --member="serviceAccount:$[e-mail]" \
  --role="roles/alloydb.client"

Configure Private services access

Private services access is implemented as a VPC peering connection between your VPC network and the underlying Google Cloud VPC network where your AlloyDB resources (clusters and instances) reside. The private connection enables resources in your VPC network to communicate with the AlloyDB resources they access exclusively by using internal IP addresses. The resources in your VPC network don't need Internet access or external IP addresses to reach AlloyDB resources.

To create a private services access configuration in a VPC network that resides in the same Cloud project as AlloyDB, you perform two operations:

Create an allocated IP address range in the VPC network.

gcloud compute addresses create google-managed-services-default \
    --global \
    --purpose=VPC_PEERING \
    --prefix-length=20 \
    --network=projects/$PROJECT_ID/global/networks/default

Create a private connection between the VPC network and the underlying Google Cloud VPC network. This step takes ~2 minutes.

gcloud services vpc-peerings connect \
    --service=servicenetworking.googleapis.com \
    --ranges=google-managed-services-default \
    --network=default \
    --project=$PROJECT_ID

Create an AlloyDB cluster.

gcloud beta alloydb clusters create $CLUSTER \
    --password=$DB_PASSWORD \
    --network=default \
    --region=$REGION \
    --project=$PROJECT_NAME

gcloud beta alloydb clusters describe $CLUSTER --region=$REGION

Create an AlloyDB instance. This step takes ~10 minutes.

gcloud beta alloydb instances create $INSTANCE \
    --cluster=$CLUSTER \
    --region=$REGION \
    --instance-type=PRIMARY \
    --cpu-count=2 \
    --project=$PROJECT_NAME

gcloud beta alloydb instances describe $INSTANCE \
    --cluster=$CLUSTER \
    --region $REGION

Review created cluster in the Cloud console.

Save database IP address into a variable.""

export DB_HOST=$(gcloud beta alloydb instances describe $INSTANCE \
    --cluster=$CLUSTER \
    --region=$REGION \
    --format=json | jq \
    --raw-output ".ipAddress")

echo "DB_HOST=$DB_HOST"

4. Explore Cloud Run Job code

There are multiple ways to create an AlloyDB database. One is to create a Compute Engine VM, install a psql client and then connect to the instance to create a database. You can read more about this approach here.

In this lab, you will create a new AlloyDB database using Cloud Run Job.

Review files below:

• Dockerfile - installs required dependencies(postgresql-client)
• script.sh - uses command line utilities to create a database

In the Cloud Shell, create new folder:

mkdir ~/alloy-db-cloud-run-job
cd ~/alloy-db-cloud-run-job 

Create new file Dockerfile:

cat > Dockerfile <<EOF
FROM ubuntu:latest
RUN apt-get update && apt-get install -y postgresql-client && apt-get clean
COPY script.sh /
RUN chmod +x /script.sh
CMD ["/script.sh"]
ENTRYPOINT ["/bin/bash"]
EOF

Create new file script.sh with content below:

echo "Connecting to $DB_HOST"
createdb -h $DB_HOST -p 5432 $PGDB
echo "Created $PGDB database"

psql -h $DB_HOST -l

Change permission on script.sh:

chmod +x script.sh

5. Deploy Cloud Run Job

Configure Serverless VPC access. This allows Cloud Run Job to communicate with AlloyDB cluster using internal/private IP. This step takes ~2 minutes.

gcloud compute networks vpc-access connectors create ${VPC_CONNECTOR} \
    --region=${REGION} \
    --range=10.8.0.0/28

Create Artifact Registry repository to store container images.

gcloud artifacts repositories create db-job-repository \
  --repository-format=docker \
  --location=$REGION

Build and publish container image to Artifact Registry.

gcloud builds submit -t $REGION-docker.pkg.dev/$PROJECT_NAME/db-job-repository/db-job:latest

Deploy Cloud Run Job.

gcloud beta run jobs create db-job \
    --image=$REGION-docker.pkg.dev/$PROJECT_NAME/db-job-repository/db-job:latest \
    --set-env-vars DB_HOST=$DB_HOST \
    --set-env-vars PGUSER=$DB_USER \
    --set-env-vars PGPASSWORD=$DB_PASSWORD \
    --set-env-vars PGDB=$DB_DATABASE \
    --vpc-connector $VPC_CONNECTOR \
    --region $REGION

Review flags that were used to create the job:

--vpc-connector - Cloud Run Job will use VPC connector to reach AlloyDB Private IP address.

Review created job in the Cloud console.

Execute Cloud Run Job to create test database.

gcloud beta run jobs execute db-job --region $REGION

Review Cloud Run Job logs in the Cloud Console.

6. Congratulations!

Congratulations, you finished the codelab!

What we've covered:

• How to create AlloyDB cluster and instance
• How to create AlloyDB database using Cloud Run Job

Clean up

To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, either delete the project that contains the resources, or keep the project and delete the individual resources.

Deleting the project

The easiest way to eliminate billing is to delete the project that you created for the tutorial.