How to Schedule a Cloud Run Job within a VPC Service Controls perimeter

How to Schedule a Cloud Run Job within a VPC Service Controls perimeter

About this codelab

subjectLast updated Jan 18, 2025
account_circleWritten by a Googler

1. Introduction

Overview

If your GCP project is inside a VPC SC perimeter, you'll need to take additional steps in order to set up scheduled jobs. Because Cloud Scheduler cannot directly trigger jobs within VPC SC perimeters, you will need to proxy the request through another component. We recommend using a Cloud Run service as this proxy.

The architecture looks like this:

Diagram of Cloud Scheduler triggering a Cloud Run service that executes a Cloud Run job

What you'll learn

  • How to run a Cloud Run Job on a schedule inside a VPC SC perimeter
  • How to create a Cloud Run service that triggers a Cloud Run job using the Cloud Run client libraries
  • How to configure Cloud Scheduler to invoke a Cloud Run service on a schedule

2. Before you begin

First, make sure you have followed the steps to set up Cloud Run for VPC Service Controls.

Next, set environment variables that will be used throughout this codelab.

PROJECT_ID=<YOUR_PROJECT_ID>
REGION=<YOUR_REGION>
AR_REPO=sample-job-repo
CLOUD_RUN_SERVICE=job-runner-service
CLOUD_RUN_JOB=sample-job
CLOUD_SCHEDULER=job-scheduler
SERVICE_ACCOUNT="cloud-run-invoker-sa"
SERVICE_ACCOUNT_ADDRESS=$SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com
NETWORK=default
SUBNET=default

3. Create the Cloud Run job

This codelab uses the sample Cloud Run jobs container.

First, create an Artifact Registry repo for your Cloud Run job container.

gcloud artifacts repositories create $AR_REPO --repository-format=docker --location=$REGION --description="codelab for Cloud Run jobs on schedule within VPC SC"

Next, copy the sample Cloud Run jobs container to an Artifact Registry within your VPC SC-configured project. You can use the gcrane tool to do this by following these installation instructions. You can learn more about gcrane in the documentation for copying images between repositories.

gcrane cp us-docker.pkg.dev/cloudrun/container/job:latest $REGION-docker.pkg.dev/$PROJECT_ID/$AR_REPO/$CLOUD_RUN_JOB:latest

Second, deploy your Cloud Run job that is complaint for VPC Service Controls.

gcloud run jobs create $CLOUD_RUN_JOB --region $REGION \
 --image $REGION-docker.pkg.dev/$PROJECT_ID/$AR_REPO/$CLOUD_RUN_JOB:latest \
 --network=$NETWORK \
 --subnet=$SUBNET \
 --vpc-egress=all-traffic

For more information on creating Jobs, please follow the steps listed here in the Cloud Run jobs documentation.

4. Create a Service account

This service account will be used by Cloud Run to call the Cloud Run job.

First, create the service account by running this command:

gcloud iam service-accounts create $SERVICE_ACCOUNT \
  --display-name="Cloud Run to run a Cloud Run job"

Second, grant the Cloud Run Invoker role and the Cloud Run Viewer role to the service account.

gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member serviceAccount:$SERVICE_ACCOUNT_ADDRESS \
  --role=roles/run.invoker

gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member serviceAccount:$SERVICE_ACCOUNT_ADDRESS \
  --role=roles/run.viewer

5. Create the Cloud Run service

In this step, you'll deploy a Cloud Run service that acts as a proxy.

mkdir job-runner-service && cd $_

Create a file called main.py with the following code.

import os
from flask import Flask
app = Flask(__name__)

# pip install google-cloud-run
from google.cloud import run_v2

@app.route('/')
def hello():

    client = run_v2.JobsClient()

    # UPDATE TO YOUR JOB NAME, REGION, AND PROJECT ID
    job_name = 'projects/YOUR_PROJECT_ID/locations/YOUR_JOB_REGION/jobs/YOUR_JOB_NAME' 

    print("Triggering job...")
    request = run_v2.RunJobRequest(name=job_name)
    operation = client.run_job(request=request)
    response = operation.result()

    print(response)
    return "Done!"

if __name__ == '__main__':
    app.run(debug=True, host="0.0.0.0", port=int(os.environ.get("PORT", 8080)))

Create a file named requirements.txt with following code.

google-cloud-run
flask

Lastly, create a Dockerfile.

FROM python:3.9-slim-buster
# for logging purposes
ENV PYTHONUNBUFFERED=True

WORKDIR /app

COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt

COPY . .

CMD ["python3", "main.py"]

Next, build the container using the following docker commands. Note that Source-based deployments can be challenging to set up in a VPC SC environment. If you have an existing build and deploy pipeline, use it to build the source code into a container and deploy the container as a Cloud Run service.

docker build -t $CLOUD_RUN_SERVICE .

docker tag $CLOUD_RUN_SERVICE $REGION-docker.pkg.dev/$PROJECT_ID/$AR_REPO/$CLOUD_RUN_SERVICE

docker push $REGION-docker.pkg.dev/$PROJECT_ID/$AR_REPO/$CLOUD_RUN_SERVICE

Now deploy your Cloud Run service that is complaint for VPC Service Controls.

gcloud run deploy $CLOUD_RUN_SERVICE --region $REGION \
 --image $REGION-docker.pkg.dev/$PROJECT_ID/$AR_REPO/$CLOUD_RUN_SERVICE \
 --service-account $SERVICE_ACCOUNT_ADDRESS \
 --network=$NETWORK \
 --subnet=$SUBNET \
 --vpc-egress=all-traffic \
 --no-allow-unauthenticated \
 --ingress internal

Save the service endpoint URL by running this command:

SERVICE_URL=$(gcloud run services describe $CLOUD_RUN_SERVICE --region $REGION --format 'value(status.url)')

6. Create a Cloud Schedule job to trigger the service

# create the Cloud Scheduler job
gcloud scheduler jobs create http $CLOUD_SCHEDULER \
  --location=$REGION \
  --schedule="0 0 1 * *" \
  --uri=$SERVICE_URL \
  --http-method=GET \
  --oidc-service-account-email=$SERVICE_ACCOUNT_ADDRESS

Once the Cloud Scheduler Job is created, you can run the following command to run the Cloud Scheduler Job immediately for testing purposes:

gcloud scheduler jobs run $CLOUD_SCHEDULER --location=$REGION

NOTE:

You may need to wait a couple of minutes for the job execution to complete. You can track its status on the Cloud Run Scheduler page.

Verify the Cloud Run job ran successfully by running the following commands:

EXECUTION_NAME=$(gcloud run jobs describe $CLOUD_RUN_JOB --region $REGION --format 'value(status.latestCreatedExecution.name)')

gcloud run jobs executions describe $EXECUTION_NAME --region $REGION

and you should see something similar to the following:

✔ Execution sample-job-w6hrj in region us-central1
1 task completed successfully
Elapsed time: 28 seconds

7. Congratulations!

Congratulations for completing the codelab!

What we've covered

  • How to run a Cloud Run Job on a schedule inside a VPC SC perimeter
  • How to create a Cloud Run service that triggers a Cloud Run job using the Cloud Run client libraries
  • How to configure Cloud Scheduler to invoke a Cloud Run service on a schedule

8. Clean up

To avoid inadvertent charges, (for example, if the Cloud Run services are inadvertently invoked more times than your monthly Cloud Run invokement allocation in the free tier), you can either delete the GCP services or delete the project you created in Step 2.

To delete the Cloud Run service and Cloud Run job, go to the Cloud Run Cloud Console at https://console.cloud.google.com/run and delete the service.

If you choose to delete the entire project, you can go to https://console.cloud.google.com/cloud-resource-manager, select the project you created in Step 2, and choose Delete. If you delete the project, you'll need to change projects in your Cloud SDK. You can view the list of all available projects by running gcloud projects list.