Data to Generative AI with Spanner and Vertex AI Imagen API

1. Introduction

In this codelab, build an AI-powered pose generator using Java, Spring Boot, Cloud Spanner database and Vertex AI Imagen API. The user will input a prompt, and the application will generate a pose based on that prompt. We will also use data from the Spanner database exposed as a REST API. It's a fun and educational way to demonstrate the capabilities of Generative AI with Spring Boot on Google Cloud.

The list of services used are:

Cloud Spanner
Vertex AI Imagen API
Cloud Run

High level flow diagram

What you'll build

You'll create

A Java Spring Boot application for the Spanner data as a service API
A Java Spring Boot application for the image generation use case using Imagen
An interactive user interface for the prompt input and response

2. Requirements

A browser, such as Chrome or Firefox
A Google Cloud project with billing enabled

Before you begin

In the Google Cloud Console, on the project selector page, select or create a Google Cloud project
Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project
Make sure all the necessary APIs (Cloud Spanner API, Vertex AI API, Cloud Run API, Cloud Functions API) are enabled
You will use Cloud Shell, a command-line environment running in Google Cloud that comes pre-loaded with bq. Refer documentation for gcloud commands and usage

From the Cloud Console, click Activate Cloud Shell on the top right corner:

If your project is not set, use the following command to set it:

gcloud config set project <YOUR_PROJECT_ID>

Navigate to the Cloud Spanner page with your active Google Cloud project to get started

3. Preparing data with Spanner

Before creating the app, let's complete the database setup by creating a Cloud Spanner instance, database and table. You can refer to this blog to read more in detail about Cloud Spanner features, DDL, DML and more. You can follow the steps below to create the database objects required for this project:

Create an instance by clicking CREATE INSTANCE on the instances page.
Enter the details as shown in the following image and click CREATE:

Once created, from the instance overview page, click CREATE DATABASE
Provide the name of the database as "first-spanner-db" and enter the following DDL in the DDL TEMPLATES section and click SUBMIT:

CREATE TABLE Yoga_Poses(
  Pose_Id   INT64 NOT NULL,
  Name  STRING(1024),
  Breath STRING(1024),
  Description STRING(1024)
) PRIMARY KEY(Pose_Id);

The database and table should be created as a result of the last step. Now let's insert a few rows into the Yoga_Poses table so we can go about building our Spring Boot application.

Click Spanner Studio on the Database pane on the left and open a new query editor tab as shown in the image below:

Run the following INSERT queries:

INSERT INTO Yoga_Poses(Pose_Id, Name, Breath, Description)
VALUES(1, 'STAND', 'INHALE AND EXHALE', 
'STAND WITH YOUR FEET HIP WIDTH APART AND ARMS RESTING BY THE SIDES');

INSERT INTO Yoga_Poses(Pose_Id, Name, Breath, Description)
VALUES(2, 'PLANK', 'INHALE OR EXHALE', 
'PLANT YOUR TOES AND PALMS ON THE MAT WITH BODY PARALLEL TO THE GROUND');

INSERT INTO Yoga_Poses(Pose_Id, Name, Breath, Description)
VALUES(3, 'SIT', 'INHALE AND EXHALE', 
'SIT ON THE FLOOR LEGS CROSSED');

INSERT INTO Yoga_Poses(Pose_Id, Name, Breath, Description)
VALUES(4, 'BEND', 'EXHALE', 
'FOLD FORWARD AS YOU STAND, HANDS REACHING TO THE FLOOR');

INSERT INTO Yoga_Poses(Pose_Id, Name, Breath, Description)
VALUES(5, 'PUSH UP', 'EXHALE', 
'PLANK WITH ELBOWS ON MAT');

INSERT INTO Yoga_Poses(Pose_Id, Name, Breath, Description)
VALUES(6, 'SEATED FORWARD BEND', 'EXHALE', 
'FOLD FORWARD AS YOU SIT, HANDS TRYING TO REACH THE FEET');

INSERT INTO Yoga_Poses(Pose_Id, Name, Breath, Description)
VALUES(7, 'LUNGE', 'EXHALE', 
'ONE LEG TO THE FRONT 90 DEGREES TO THE FLOOR AND THE BACK LEG STRAIGHT');

INSERT INTO Yoga_Poses(Pose_Id, Name, Breath, Description)
VALUES(8, 'COURTESY LUNGE', 'INHALE', 
'ONE LEG TO THE FRONT 90 DEGREES TO THE FLOOR AND THE BACK KNEE TOUCHING THE FLOOR');

INSERT INTO Yoga_Poses(Pose_Id, Name, Breath, Description)
VALUES(9, 'BANK BEND', 'INHALE', 
'STAND WITH ARMS UP AND BODY BENT BACKWARDS, ARCHING YOUR SPINE, LOOKING AT THE SKY');

INSERT INTO Yoga_Poses(Pose_Id, Name, Breath, Description)
VALUES(10, 'BICEP ACTION', 'INHALE AND EXHALE', 
'CURL, PRESS AND WORK YOUR BICEPS');

Now we have our Spanner instance, database, table and data created and ready for the application.

4. Build a REST API with Spring Boot and Spanner for pose data

To familiarize yourself with Spring Boot and Google Cloud console do check out this blog.

Bootstrap the Spring Boot app To bootstrap this app from scratch, follow this blog. To clone from the repo and swing into action, just run the below command in your cloud shell terminal:

git clone https://github.com/AbiramiSukumaran/spanner-springb

This should have created the project spanner-springb in your cloud shell machine. To understand the components of the app, go through the blog.

Let's build and run the app To do this, run the following commands from the cloud shell terminal:

cd spanner-springb

./mvnw package

./mvnw spring-boot:run

Deploy the app in Cloud Run so it available for the pose generator app To do this, run the following command from the cloud shell terminal:

gcloud run deploy –source .

Pass the required parameters and make sure your app is deployed. The generated service URL should be able to list the data created in Spanner.

5. Bootstrap a pose generator app with Spring Boot, Spanner and Imagen

At this point, we have completed the top part of the flow diagram, that is, making the Spanner data available as a service (REST API). Now let's create a client application that invokes the spanner data, calls the IMAGEN API with a prompt and returns the Base64 encoded string as an image to the user interface.

By now, you are already familiar with the Spring Boot project structure and its significance. So will quickly jump straight into the cloning the repo into your cloud shell machine by running the command below in your cloud shell terminal:

git clone https://github.com/AbiramiSukumaran/genai-posegen

This should have created the project genai-posegen in your cloud shell machine. The cloned project structure shows up like this in the cloud shell editor:

PromptController Java class has the database service invocation, implementation of the business logic and the generative AI API invocation of Imagen as well. This class interacts with the Thymeleaf templates that take care of data integration to the user interface. There are 3 service methods in this class — 1) for getting the prompt input 2) for processing the request and invoking the Imagen API and 3) for processing imagen response.

Prompt and Yoga are the POJO classes that contain the fields, getters and setters to interface with the Imagen API and Spanner data server API respectively.

Index and getImage html files in the templates folder contain the templates for user interface and they have dependencies of JS and css scripts in the respective folders.

Vertex AI Imagen API Integration For the image generation use case we are using the Vertex AI's Imagen API in the following format:

https://<<region>>- aiplatform.googleapis.com/v1/projects/<<your-project-id>>/locations/<<region>>/publishers/google/models/imagegeneration:predict

You can read more about Imagen capabilities here. It returns the response in Base64 encoded string format. To convert it into its image, we have used the javascript setattribute method (in the getImage.js file) on the image object as follows in the getImage.HTML file:

poseImage.setAttribute('src', "data:image/jpg;base64," + baseStr64);

Authorization The Imagen API requires you to have bearer token authentication enabled to access it. In our case, I have used the Application Default Credentials JSON approach. You can implement it by running the below command from the cloud shell terminal and following the instructions that follow in the terminal:

gcloud auth application-default login

Enter "Y" to authenticate with your account. Allow access and copy the authorization code that is shown in the pop-up. As soon as you do that, you will get the application default credentials in the JSON file saved to a location similar to this: /tmp/tmp.Fh0Gf4yF0V/application_default_credentials.json.

Download the file or copy the contents of the JSON file by running the cat command (cat /tmp/tmp.Fh0Gf4yF0V/application_default_credentials.json) and use it in the application in the callImagen() method of the PromptController.java class. You can read more about authentication, here.

User Interface We have used Thymeleaf as the template engine to parse and render data to the front end template files and to add elegant design to the user interface. It is similar to HTML but supports more attributes to work with rendered data. The index.html contains the landing page design components and it allows the user to select the topic and add an overriding prompt to generate the desired image.

6. Build and deploy

Now that you have cloned the code, replaced the values for placeholders as applicable to your project, region and authentication credentials, let's move on to building and deploying the app. Navigate to the project folder in the cloud shell terminal using the command, the build and then deploy for test locally in the cloud shell machine with the following 3 commands one by one:

cd genai-posegen

./mvnw package

./mvnw spring-boot:run

Deploy the app in Cloud Run so it is available on the cloud. To do this, run the following command from the cloud shell terminal within the project folder:

gcloud run deploy –source .

Pass the required parameters and make sure your app is deployed.

7. Demo

Once the app is deployed, you should see the service URL in the terminal. Click the link and see your pose image generation app running on Google Cloud serverlessly!

8. Clean up

To avoid incurring charges to your Google Cloud account for the resources used in this post, follow these steps:

In the Google Cloud console, go to the Manage resources page
In the project list, select the project that you want to delete, and then click Delete
In the dialog, type the project ID, and then click Shut down to delete the project
If you don't want to delete the project, delete the Spanner instance by navigating to the instance you just created for this project and click the DELETE INSTANCE button on the top right corner of the instance overview page
You can also navigate to the Cloud Run services page, and select the services created in this project and click the delete button to delete the services.

9. Congratulations

In this blog, we were able to bring the full stack Spring Boot application that stores and handles data in Cloud Spanner, to generate poses using Google Cloud Vertex AI's Imagen API in an interactive client application deployed in Cloud Run. In the architecture diagram section of this blog, you see the Cloud Functions (Java) component which we never got to do? That is up for grabs if you wish to contribute. You can implement 2 Java Cloud Functions to perform the 2 methods that can be found in the getimage.html file: save pose to database and upload image methods. Check out Imagen on Vertex AI documentation to learn more about the model.