1. Overview
In this lab, you will learn how to use Cloud Storage bucket events to trigger Cloud Functions to analyze data and process images. The function will use Google's Vision API and save the resulting image information as object metadata back in Cloud Storage.
What you will learn
How to build an image processing pipeline.
- Configure Storage buckets
- Create a Cloud Function to read and write objects in Cloud Storage
- Integrate Vision API to detect food images
- Deploy a Cloud Function
- Test and validate the end-to-end solution
Prerequisites
- This lab assumes familiarity with the Cloud Console and shell environments.
- Prior Cloud Storage, Cloud Function, or Vision API experience is helpful but not required.
2. Setup and Requirements
Self-paced environment setup
- Sign-in to the Google Cloud Console and create a new project or reuse an existing one. If you don't already have a Gmail or Google Workspace account, you must create one.
- The Project name is the display name for this project's participants. It is a character string not used by Google APIs, and you can update it at any time.
- The Project ID must be unique across all Google Cloud projects and is immutable (cannot be changed after it has been set). The Cloud Console auto-generates a unique string; usually you don't care what it is. In most codelabs, you'll need to reference the Project ID (and it is typically identified as
PROJECT_ID
), so if you don't like it, generate another random one, or, you can try your own and see if it's available. Then it's "frozen" after the project is created. - There is a third value, a Project Number which some APIs use. Learn more about all three of these values in the documentation.
- Next, you'll need to enable billing in the Cloud Console in order to use Cloud resources/APIs. Running through this codelab shouldn't cost much, if anything at all. To shut down resources so you don't incur billing beyond this tutorial, follow any "clean-up" instructions found at the end of the codelab. New users of Google Cloud are eligible for the $300 USD Free Trial program.
Start Cloudshell Editor
This lab was designed and tested for use with Google Cloud Shell Editor. To access the editor,
- Access your google project at https://console.cloud.google.com.
- In the top right corner click on the cloud shell editor icon
- A new pane will open in the bottom of your window
- Click on the Open Editor button
- The editor will open with an explorer on the right and editor in the central area
- A terminal pane should also be available in the bottom of the screen
- If the terminal is NOT open use the key combination of ctrl+` to open a new terminal window
Environment Setup
- Create project and resource-related environment variables
export PROJECT_ID=$(gcloud config get-value project)
export PROJECT_NAME=$(gcloud config get-value project)
export REGION=us-east4
export UPLOAD_BUCKET=gs://menu-item-uploads-$PROJECT_ID
export BUCKET_THUMBNAILS=gs://menu-item-thumbnails-$PROJECT_ID
export MENU_SERVICE_NAME=menu-service
export USER_EMAIL=$(gcloud config list account --format "value(core.account)")
- Enable the APIs required for the lab
gcloud services enable \
cloudbuild.googleapis.com \
vision.googleapis.com \
cloudfunctions.googleapis.com
- Clone the repository
git clone https://github.com/momander/cymbal-eats.git && cd cymbal-eats/menu-service
- Deploy the menu services
./setup.sh
Done. Service [menu-service] revision [menu-service-00001-dok] has been deployed and is serving 100 percent of traffic.
- Store the menu service's URL
MENU_SERVICE_URL=$(gcloud run services describe $MENU_SERVICE_NAME \
--region=$REGION \
--format=json | jq \
--raw-output ".status.url")
echo $MENU_SERVICE_URL
(Example Output)
https://menu-service-eb2ckzn7hq-uk.a.run.app
- Navigate out of the menu-service directory
cd ../cloud-functions
3. Configuring Cloud Storage buckets
Create Storage Buckets
Create upload and output Cloud Storage buckets for your image processing pipeline.
- Use the gsutil mb command and a unique name to create a bucket to download images:
gsutil mb -p $PROJECT_ID -l $REGION $BUCKET_THUMBNAILS
- Create a bucket to upload images
gsutil mb -p $PROJECT_ID -l $REGION $UPLOAD_BUCKET
Update Bucket permissions
Update the storage bucket permissions to allow read and write permissions to users.
- Use the gsutil iam ch command to give permission to read and write objects in your bucket:
gsutil iam ch allUsers:objectViewer $BUCKET_THUMBNAILS
Warning: It is best practice to limit access to storage buckets. For the purpose of this lab, all users will be allowed to view all objects. For more information on securing storage buckets review Use IAM Permission and Best Practices for Cloud Storage
- Grant the storage object creator role to the service account to view and write objects
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:$PROJECT_ID@appspot.gserviceaccount.com" \
--role="roles/storage.objectCreator"
4. Creating a Function
Create a function to download an image from Cloud Storage, resize the image and upload the image back to Cloud Storage. The function will call the Vision API to assign a description label to the image. The function will check the description label. If the label identifies the image as "Food" a call will be made to the menu service to update the menu item's image and thumbnail.
Triggering a Function
Currently, Cloud Storage functions are based on Pub/Sub notifications from Cloud Storage and support similar event types:
In this lab will describe how to deploy and trigger a function when an object is finalized
Object Finalize
Object finalize events trigger when a "write" of a Cloud Storage Object is successfully finalized. In particular, this means that creating a new object or overwriting an existing object triggers this event. Archive and metadata update operations are ignored by this trigger.
- Navigate to the starter-code directory.
cd starter-code
- Open Cloud Editor
cloudshell edit package.json index.js
5. Integrating Cloud Storage
Cloud Storage is a service for storing your objects in Google Cloud. An object is an immutable piece of data consisting of a file of any format. You store objects in containers called buckets. All buckets are associated with a project, and you can group your projects under an organization.
- In the starter package.json file, add the Cloud Storage dependency by replacing the "TODO 1": "Add the Cloud Storage dependency" with the following snippet
starter-code/package.json
"@google-cloud/storage": "^5.18.2",
- In the starter index.js file, add the storage modules by replacing the //1. TODO... with the following snippet
starter-code/index.js
const {Storage} = require('@google-cloud/storage');
- In the starter index.js file, create a Storage object by replacing the //2. TODO... with the following snippet
starter-code/index.js
const storage = new Storage();
- In the starter index.js file, create a reference to the Cloud Storage buckets by replacing the //3. TODO... with the following snippet
starter-code/index.js
const bucket = storage.bucket(file.bucket);
const thumbBucket = storage.bucket(process.env.BUCKET_THUMBNAILS);
- In the starter index.js file, download a file into memory or to a local destination by replacing the //4. TODO... with the following snippet
Download Cloud Storage objects
starter-code/index.js
await bucket.file(file.name).download({
destination: originalFile
});
- In the starter index.js file, get the public URL of this file by replacing the //5. TODO... with the following snippet
starter-code/index.js
const originalImageUrl = await bucket.file(file.name).publicUrl();
Upload objects to Cloud Storage
- In the starter index.js file, upload a file to the bucket and get the public URL of the file by replacing the //6. TODO... with the following snippet
starter-code/index.js
const thumbnailImage = await thumbBucket.upload(thumbFile);
const thumbnailImageUrl = thumbnailImage[0].publicUrl();
6. Integrating Vision API
Cloud Vision allows developers to easily integrate vision detection features within applications, including image labeling, face and landmark detection, optical character recognition (OCR), and tagging of explicit content.
- In the starter package.json file, add the Vision API dependency by replacing the "TODO 2": "Add the Vision API dependency" with the following snippet
starter-code/package.json
"@google-cloud/vision": "^2.4.2",
- In the starter index.js file, add the vision API module by replacing the //6. TODO... with the following snippet
starter-code/index.js
const vision = require('@google-cloud/vision');
- In the starter index.js file, construct an instance of Image Annotator Client by replacing the //7. TODO... with the following snippet
Create a Image Annotator Client
starter-code/index.js
const client = new vision.ImageAnnotatorClient();
Build a Vision API request
- In the starter index.js file, build a vision request by replacing the //8. TODO... with the following snippet
starter-code/index.js
const visionRequest = {
image: { source: { imageUri: `gs://${file.bucket}/${file.name}` } },
features: [
{ type: 'LABEL_DETECTION' },
]
};
- In the starter index.js file, annotate the image with the requested features by replacing the //9. TODO... with the following snippet
Execute a Vision API request
starter-code/index.js
const visionPromise = client.annotateImage(visionRequest);
- In the starter index.js file, loop through label annotations and set status to Ready if the image has a "Food" label by replacing the //10. TODO... with the following snippet
starter-code/index.js
for (label of visionResponse[0].labelAnnotations){
status = label.description == "Food" ? "Ready" : status
}
7. Deploying the Function and Create the trigger
- Click on Open Terminal
- If you are in the process_image folder - navigate one level up: cd ..
- To deploy your Cloud Function with a storage trigger on the
menu-item-uploads-$PROJECT_ID
bucket
gcloud functions deploy process_thumbnails \
--region=$REGION \
--trigger-resource=$UPLOAD_BUCKET \
--trigger-event=google.storage.object.finalize \
--runtime=nodejs16 \
--set-env-vars=BUCKET_THUMBNAILS=$BUCKET_THUMBNAILS,MENU_SERVICE_URL=$MENU_SERVICE_URL \
--source=process_image \
--project=$PROJECT_ID
- Verify the Cloud Function
gcloud functions list
(Example Output)
NAME: process_thumbnails STATUS: ACTIVE TRIGGER: Event Trigger REGION: us-east4
8. Test and validate the end-to-end solution
Upload a new photo to Cloud Storage and monitor the progress of the pipeline as the images are analyzed and posted to the menu service. You will test the end-to-end solution by updating the image for one of the Menu Services food items.
Adding an appropriate image
- Upload an image to the upload bucket to replace menu item 1's image
curl https://unsplash.com/photos/9eIbwtyl4Xs | gsutil cp 1.jpeg ${UPLOAD_BUCKET}
Copying file://1.jpeg [Content-Type=image/jpeg]... / [1 files][191.0 KiB/191.0 KiB] Operation completed over 1 objects/191.0 KiB.
- In Cloud Console, navigate to Cloud Functions
- Click on process_thumbails
- Click on the LOGS tab
The function ran successfully and loaded the image to the thumbnails bucket
- Navigate into the menu-item-thumbnails-$PROJECT_ID Cloud Storage bucket
- Verify that the image has been uploaded to the bucket
- Click on the image name, this will open the image details page to view the image URL
- Open cloudshell and query the menu service to check the status
curl -X get ${MENU_SERVICE_URL}/menu/1| python3 -m json.tool
Example Output
{ "id": 1, "createDateTime": "2022-04-13T16:02:50.322969", "itemImageURL": "https://storage.googleapis.com/menu-item-uploads-cymbal-eats-23375/1.jpeg", "itemName": "Curry Plate", "itemPrice": 12.5, "itemThumbnailURL": "https://storage.googleapis.com/menu-item-thumbnails-cymbal-eats-23375/1.jpeg", "spiceLevel": 0, "status": "Ready", "tagLine": "Spicy touch for your taste buds!!", "updateDateTime": "2022-04-13T17:58:57.675079" }
- Verify the itemImageURL from cloudshell and the Authenticated URL match
Adding an inappropriate image
To verify the function works properly you will upload an image that does not contain an object that would be classified as a "Food" item.
- Upload an image to the upload bucket to replace menu item 2's image
curl https://unsplash.com/photos/w2JtIQQXoRU | gsutil cp 2.jpeg ${UPLOAD_BUCKET}
- Query the menu service for menu item 2
curl -X get ${MENU_SERVICE_URL}/menu/2| python3 -m json.tool
Example Output
{ "id": 2, "createDateTime": "2022-04-17T23:53:15.827274", "itemImageURL": "https://storage.googleapis.com/menu-item-uploads-cymbal-eats-18147-25762/2.jpeg", "itemName": "Gulab Jamoon", "itemPrice": 2.4, "itemThumbnailURL": "https://storage.googleapis.com/menu-item-thumbnails-cymbal-eats-18147-25762/2.jpeg", "spiceLevel": 0, "status": "Failed", "tagLine": "Sweet cottage cheese dumplings", "updateDateTime": "2022-04-18T02:13:55.348545" }
9. Congratulations!
Congratulations, you finished the codelab!