Triggering Cloud Run Jobs with Cloud Scheduler

1. Overview

Cloud Run jobs feature is covered by the Pre-GA Offerings Terms of the Google Cloud Terms of Service. Pre-GA features might have limited support, and changes to pre-GA features might not be compatible with other pre-GA versions. For more information, see the launch stage descriptions.

In this lab, you will create a Cloud Run job and set up a Cloud Scheduler job. You will deploy Cymbal Eats Menu Service using the setup script. You will create a Cloud Run job that makes API calls to Cymbal Eats Menu Service. You will execute the job using Google Cloud CLI and set up a schedule for the job. You will verify execution by reviewing logs and making API calls to Menu Service to confirm that menu items were deleted.

What are Cloud Run jobs?

Cloud Run job runs a container that does not serve web requests but instead executes operational tasks or data processing. The container will run the task and exit when finished.

Cleanup service job

The cleanup service job will retrieve menu items in the Failed status and delete them. When new menu items are created, images are analyzed using Vision API to detect if it's a food item or not. For images that fail this validation, menu items status will be updated to Failed and subsequently deleted by the cleanup job.

What you will learn

In this lab, you will learn how to do the following:

• Create Cloud Run jobs
• Execute Cloud Run jobs
• Create Cloud Scheduler jobs
• Verify jobs execution

Prerequisites

• This lab assumes familiarity with the Cloud Console and shell environments.
• Prior Cloud Run and Cloud Scheduler experience is helpful but not required.

2. Setup and Requirements

Cloud Project setup

1. Sign-in to the Google Cloud Console and create a new project or reuse an existing one. If you don't already have a Gmail or Google Workspace account, you must create one.

• The Project name is the display name for this project's participants. It is a character string not used by Google APIs. You can update it at any time.
• The Project ID is unique across all Google Cloud projects and is immutable (cannot be changed after it has been set). The Cloud Console auto-generates a unique string; usually you don't care what it is. In most codelabs, you'll need to reference the Project ID (it is typically identified as PROJECT_ID). If you don't like the generated ID, you may generate another random one. Alternatively, you can try your own and see if it's available. It cannot be changed after this step and will remain for the duration of the project.
• For your information, there is a third value, a Project Number which some APIs use. Learn more about all three of these values in the documentation.

Caution: A project ID is globally unique and cannot be used by anyone else after you've selected it. You are the only user of that ID. Even if a project is deleted, the ID can never be used again

Note: If you're using a Gmail account, you can leave the default location set to No organization. If you're using a Google Workspace account, then choose a location that makes sense for your organization.

2. Next, you'll need to enable billing in the Cloud Console to use Cloud resources/APIs. Running through this codelab shouldn't cost much, if anything at all. To shut down resources so you don't incur billing beyond this tutorial, you can delete the resources you created or delete the whole project. New users of Google Cloud are eligible for the $300 USD Free Trial program.

Environment Setup

Activate Cloud Shell by clicking on the icon to the right of the search bar.

From the Cloud Shell, run the following command to clone the application code from this repo and go to the directory containing the menu service:

git clone https://github.com/GoogleCloudPlatform/cymbal-eats.git && cd cymbal-eats/menu-service

Deploy the Menu service using the setup script to Cloud Run. The Menu service is a Java-based microservice built with the Quarkus framework using Cloud SQL Postgres database for its backend. The Menu service is runtime dependency for the Cloud Run job you will create in the following steps.

./setup.sh

The deployment will take around 10 minutes to create all required components.

Continue with the next steps after executing the command above.

3. Explore Cloud Run Job code

Open a new tab in the Cloud Shell by clicking the plus icon.

Go to the directory containing the cleanup service and review files that make up the job:

cd ~/cymbal-eats/cleanup-service

The cleanup-service in this directory contains a Dockerfile that defines the container image for the cleanup service job with required dependencies(httpie, jq).

Dockerfile

FROM ubuntu:latest 
RUN apt-get update && apt-get install -y httpie jq && apt-get clean
COPY script.sh /
RUN chmod +x /script.sh
CMD ["/script.sh"]
ENTRYPOINT ["/bin/bash"]

The actual cleanup script, listed below, contains commands to get a list of menu items in failed status and delete them by making API calls to the Menu service.

script.sh

echo "FAILED_ITEM_AGE=$FAILED_ITEM_AGE"
echo "MENU_SERVICE_URL=$MENU_SERVICE_URL"
# Failed items older than FAILED_ITEM_AGE in minutes
for id in $(http GET $MENU_SERVICE_URL/menu/failed | jq '[.[] | select(.updateDateTime < ((now - 60 * (env.FAILED_ITEM_AGE | tonumber) )| strftime("%Y-%m-%dT%H:%M:%S.%f")))]'| jq '.[].id'); do
  echo "Deleting Menu Item : $MENU_SERVICE_URL/menu/$id"
  http GET $MENU_SERVICE_URL/menu/$id
  http DELETE $MENU_SERVICE_URL/menu/$id
done

# Processing items older than FAILED_ITEM_AGE in minutes
for id in $(http GET $MENU_SERVICE_URL/menu/processing | jq '[.[] | select(.updateDateTime < ((now - 60 * (env.FAILED_ITEM_AGE | tonumber))| strftime("%Y-%m-%dT%H:%M:%S.%f")))]'| jq '.[].id'); do
  echo "Deleting Menu Item : $MENU_SERVICE_URL/menu/$id"
  http GET $MENU_SERVICE_URL/menu/$id
  http DELETE $MENU_SERVICE_URL/menu/$id
done

Notice the following about the script:

• FAILED_ITEM_AGE and MENU_SERVICE_URL environment variables will be set during deployment and passed by Cloud Run job.
• FAILED_ITEM_AGE - Number of minutes before Failed item will be deleted.
• MENU_SERVICE_URL - Cymbal Eats Menu service url.

4. Create Cloud Run Job

Next you will build a container image and publish it to the Artifact Registry.

This container image will be used to create a Cloud Run job.

Enable service APIs:

gcloud services enable \
    run.googleapis.com \
    artifactregistry.googleapis.com \
    cloudscheduler.googleapis.com \
    --quiet

Set environment variables:

export PROJECT_ID=$(gcloud config get-value project)
export PROJECT_NUMBER=$(gcloud projects describe $PROJECT_ID --format='value(projectNumber)')
export PROJECT_NAME=$(gcloud projects describe $PROJECT_ID --format='value(name)')
export REGION=us-east1
export MENU_SERVICE_NAME=menu-service

Create a new Artifact Registry repository to store docker images for cleanup job:

gcloud artifacts repositories create cymbal-eats --repository-format=docker --location=$REGION

Build container image using Cloud Build and push it to Artifact Registry with one command:

gcloud builds submit -t $REGION-docker.pkg.dev/$PROJECT_NAME/cymbal-eats/cleanup-service:latest

Example output:

DURATION: 35S
SOURCE: gs://cymbal-eats-14906-569_cloudbuild/source/1657126400.933586-dc3e91ec85934a55bb6d2f7012611365.tgz
IMAGES: us-east1-docker.pkg.dev/cymbal-eats-14906-569/cymbal-eats/cleanup-service (+1 more)
STATUS: SUCCESS

After publishing is done, navigate to Artifact Registry and review published image:

Switch to the first Cloud Shell tab to check if Menu Service deployment is still in progress, wait until it's finished before moving to the next step.

Switch back to the second Cloud Shell tab. Run the following command to describe Menu service and save URL to environment variable. This environment variable will be used to configure the Cloud Run job.

MENU_SERVICE_URL=$(gcloud run services describe $MENU_SERVICE_NAME \
 --region=$REGION \
 --format=json | jq \
 --raw-output ".status.url")

Create a Cloud Run job to clean up failed menu items that are older than 1 minute [set by FAILED_ITEM_AGE].

gcloud beta run jobs create cleanup-service \
  --image=$REGION-docker.pkg.dev/$PROJECT_NAME/cymbal-eats/cleanup-service:latest \
  --set-env-vars MENU_SERVICE_URL=$MENU_SERVICE_URL \
  --set-env-vars FAILED_ITEM_AGE=1 \
  --region $REGION

Example output:

Creating Cloud Run job [cleanup-service] in project [cymbal-eats] region [us-east1]
OK Creating job... Done.
Done.
Job [cleanup-service] has successfully been created.

Navigate to the Cloud Run JOBS section in the console and review the created job.

Click on the job and explore available tabs: HISTORY, LOGS, CONFIGURATION and YAML.

Verify that environment variables were set by reviewing job's CONFIGURATION section in the console:

(Optional) If you want to change the Failed Item Age or Menu Service URL variables, after the Cloud Run job was created, you can use update command:

gcloud beta run jobs update cleanup-service \
  --image=$REGION-docker.pkg.dev/$PROJECT_NAME/cymbal-eats/cleanup-service:latest \
  --set-env-vars MENU_SERVICE_URL=$MENU_SERVICE_URL \
  --set-env-vars FAILED_ITEM_AGE=1 \
  --region $REGION

To validate the job, execute the Cloud Run job by running following command:

gcloud beta run jobs execute cleanup-service --region=$REGION

Example output:

OK Creating execution... Done.                                   
  OK Provisioning resources...
Done.
Execution [cleanup-service-rlxs4] has successfully started running.

View details about this execution by running:
gcloud beta run jobs executions describe cleanup-service-rlxs4
 

Switch to the LOGS tab to review the job's output. You should see the Failed Item Age and Menu Service URL in the logs.

5. Set Up A Schedule for Cloud Run Job

Cloud Scheduler is a fully managed enterprise-grade cron job scheduler. It allows you to schedule virtually any job, including batch, big data jobs, cloud infrastructure operations, and more.

One security best practice when working with a Cloud Scheduler job is to execute each job with separate credentials. In this step create a Service account for use by the cleanup scheduler job.

export SCHEDULER_SERVICE_ACCOUNT=cleanup-scheduler-job-sa

gcloud iam service-accounts create ${SCHEDULER_SERVICE_ACCOUNT}

Cloud Scheduler job will need permissions to make calls to Cloud Run Jobs.

Grant Cloud Run Invoker role to the service account that is used in the Cloud Scheduler job:

gcloud projects add-iam-policy-binding ${PROJECT_ID} \
  --member="serviceAccount:${SCHEDULER_SERVICE_ACCOUNT}@${PROJECT_ID}.iam.gserviceaccount.com" \
  --role="roles/run.invoker"

Next you will set up a schedule to run the cleanup service job.

There are multiple target types supported by Cloud Scheduler.

• HTTP
• Pub/Sub
• App Engine HTTP

You will create a scheduler job using HTTP target type.

For demonstration purposes you will schedule it to run every 5 minutes.

gcloud scheduler jobs create http cleanup-schedule \
    --location $REGION \
    --schedule="*/5 * * * *" \
    --uri="https://$REGION-run.googleapis.com/apis/run.googleapis.com/v1/namespaces/$PROJECT_ID/jobs/cleanup-service:run" \
    --http-method POST \
    --oauth-service-account-email ${SCHEDULER_SERVICE_ACCOUNT}@${PROJECT_ID}.iam.gserviceaccount.com

Review uri parameter that is used to call Cloud Run Job:

• REGION and PROJECT_ID - Cloud Run Region and Project ID where cleanup service job is deployed
• cleanup-service - name of the Cloud Run Job

Navigate to Cloud Scheduler in the console to review created scheduler job:

Review available options under the Actions menu.

6. Test Cloud Run Job

Using Menu Service endpoints, review existing menu items and their status:

curl ${MENU_SERVICE_URL}/menu | jq

Output:

You will see 3 menu items in Ready status.

Change status of menu item #1 to Failed:

curl -X PUT "${MENU_SERVICE_URL}/menu/1" \
  -H 'Content-Type: application/json' \
  -d '{"status": "Failed"}' | jq

Wait for 1 minute. For the menu item to be deleted, it needs to be 1 minute old, as set by FAILED_ITEM_AGE parameter.

You can wait for the next scheduled run or force job execution from the console.

There are multiple ways to trigger a job, through the UI or from the command line.

For this example, run command in the Cloud Shell(Option #3) to trigger the job.

1. From Cloud Scheduler by selecting "Force a job run" from the Actions menu.

2. From Cloud Run Job by clicking the "EXECUTE" button.

3. From Cloud Shell by running following command:

gcloud beta run jobs execute cleanup-service --region=$REGION

Navigate to the Cloud Run JOBS section, open the LOGS tab and verify that the menu item was deleted.

Filter logs for "deleting" keyword to find the logs.

Use Menu Service endpoints to check existing menu items through the REST endpoint.

curl ${MENU_SERVICE_URL}/menu | jq

Output:

You will see 2 menu items in Ready status.

7. Congratulations!

Congratulations, you finished the codelab!

What we've covered:

• How to create Cloud Run jobs
• How to execute Cloud Run jobs
• How to create Cloud Scheduler jobs
• How to verify jobs execution

What's next:

Explore other Cymbal Eats codelabs:

• Triggering Cloud Workflows with Eventarc
• Triggering Event Processing from Cloud Storage
• Connecting to Private CloudSQL from Cloud Run
• Connecting to Fully Managed Databases from Cloud Run
• Secure Serverless Application with Identity Aware Proxy (IAP)
• Securely Deploying to Cloud Run
• Securing Cloud Run Ingress Traffic
• Connecting to private AlloyDB from GKE Autopilot

Clean up

To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, either delete the project that contains the resources, or keep the project and delete the individual resources.

Deleting the project

The easiest way to eliminate billing is to delete the project that you created for the tutorial.