1. Introduction
In this codelab, I have listed the steps to build a web app that lets you perform text summarization using Vertex AI Large Language Model for text generation ( text-bison) API in a Svelte Kit framework. The list of services and tech stack used are:
- Svelte Kit: A web application framework that builds on Svelte.
- Vertex AI PaLM 2 API: A large language model (LLM) API that provides access to Google AI's PaLM 2 model.
- Cloud Functions: A serverless platform to run functions without having to manage servers.
- Cloud Run: A serverless platform to run containerized applications.
What you'll build
You'll create
- A Python Cloud Function is used to makePalm API invocation
- A Svelte web application for the user interface to interact with the Vertex AI API through the Cloud Function
- A Cloud Run service to deploy the app created above serverlessly
2. Requirements
3. Before you begin
- In the Google Cloud Console, on the project selector page, select or create a Google Cloud project
- Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project
- You will use Cloud Shell, a command-line environment running in Google Cloud to interact with the Google Cloud services. Refer documentation for gcloud commands and usage. From the Cloud Console, click Activate Cloud Shell on the top right corner:
If your project is not set, use the following command to set it:
gcloud config set project <YOUR_PROJECT_ID>
- Make sure all the necessary APIs are enabled by running the following command from the Cloud Shell Terminal:
gcloud services enable cloudbuild.googleapis.com \
run.googleapis.com \
cloudfunctions.googleapis.com \
aiplatform.googleapis.com
- Create Environment variables for REGION and PROJECT_ID by running the following command from the Cloud Shell Terminal:
export PROJECT_ID=<your project id>
export REGION=asia-south1
4. Cloud Function to invoke the Vertex AI API
We will create a Python Cloud Function and invoke the Vertex AI API in this function.
Creating a new Service Account
Create a new service account by executing the following command in the Cloud Shell Terminal.
gcloud iam service-accounts create vertex-service-acc
To provide access to your project and your resources, grant a role to the service account.
gcloud projects add-iam-policy-binding ${PROJECT_ID} --member="serviceAccount:vertex-service-acc@${PROJECT_ID}.iam.gserviceaccount.com" --role=roles/ml.developer
To grant your Google Account a role that lets you use the service account's roles and attach the service account to other resources. Replace USER_EMAIL with your Google Account Email ID.
gcloud iam service-accounts add-iam-policy-binding vertex-service-acc@${PROJECT_ID}.iam.gserviceaccount.com --member="user:USER_EMAIL" --role=roles/iam.serviceAccountUser
Creating the Python function
The PaLM 2 for text is ideal for tasks that can be completed with one API response, without the need for continuous conversation. Let's create the Cloud Function for it now.
Create a new directory using cloud shell and navigate to it. (use the same terminal opened in previous section):
mkdir vertex-ai-functions
cd vertex-ai-functions
Create a main.py file for writing Python Cloud Function and requirements.txt file for storing dependencies.
touch main.py requirements.txt
This Python file defines a simple HTTP Cloud Function that uses a Vertex AI Text Generation Model to generate short summaries of text inputs. The function takes a text input as a parameter and returns a short summary of the input. The function uses a variety of parameters to control the generation process, such as the creativity, diversity, and fluency of the generated text. The HTTP Cloud Function accepts a request object and returns the model's summary as the response.
Open Google Cloud Editor. You can open this by opening a new Google Cloud console tab and clicking on ACTIVATE CLOUD SHELL button and when the terminal loads, quickly click on the OPEN EDITOR button as shown in the image below:
Once the editor opens, you should be able to see the main.py file. Replace its content with the code from this repository link. The explanation for the code is included as code comments. In summary, this code provides a concise way to generate short summaries of text inputs using Vertex AI.
The requirements.txt file have package dependencies: functions-framework==3.*: Ensures that the function uses the latest features and bug fixes of the Functions Framework. google-cloud-aiplatform: Required to use the Vertex AI Text Generation Model.
Add this to requirements.txt file:
functions-framework==3.*
google-cloud-aiplatform
Deploy to Cloud Functions
Now let's deploy this source to Cloud Functions. Run the following command from the cloud Shell Terminal:
gcloud functions deploy vertex-ai-function \
--gen2 \
--runtime=python311 \
--region=${REGION} \
--source=. \
--entry-point=hello_vertex \
--trigger-http \
--allow-unauthenticated \
--max-instances=30
Use the search bar and go to the Cloud Functions console:
This will list the vertex-ai-function Cloud Function we just created with its public URL will be given in the function page. We use this to connect our Frontend and Vertex AI API. Store this URL. You may also have to allow the underlying Cloud Run service for unauthenticated access. Use authenticated service is recommended for security purposes.
5. Build and deploy the front-end
This app comes with a frontend interface to interact with our Vertex AI API through Google Cloud Functions. Let's create it now.
Clone Repository and Setup Dockerfile
Go to the root directory and clone the git repository.
cd ~/
git clone https://github.com/bhaaratkrishnan/vertex-summarizer-svelte.git
cd vertex-summarizer-svelte
To run this application, you need to add the PUBLIC_FUNCTION_URL environment variable in Dockerfile. This URL is the Cloud Function URL created and stored in the previous section.
Open Cloud Editor and edit the contents of Dockerfile file. Replace the PUBLIC_FUNCTION_URL variable with your Cloud Function URL.
Deploy Frontend to Cloud Run
We will be using Google Artifact Registry to build and store our Docker Images. Cloud Run is used to deploy the containers in Serverless Architecture.
Create an Artifact Registry Repository by running the command below in Cloud Shell Terminal:
gcloud artifacts repositories create vertex-repo --repository-format=docker --location=${REGION}
Create an environment variable for Artifact Registry repository URL.
export DOCKER_URL=${REGION}-docker.pkg.dev/${PROJECT_ID}/vertex-repo/vertex-summarizer-image
Build the docker container and tag it with Artifact Registry Repository location. Tagging the Docker image with a repository name configures the docker push command to push the image to a specific location.
docker build . -t ${DOCKER_URL}
Push the image to the Artifact Registry.
docker push ${DOCKER_URL}
Deploy docker container to Cloud Run.
gcloud run deploy vertex-summarizer --allow-unauthenticated --platform=managed --region=${REGION} --image=${DOCKER_URL}
Yaay !! Vertex Summarizer is up and running. The URL will be shown in Cloud Shell, so Explore and Enjoy Vertex AI🤖.
6. Clean up
To avoid incurring charges to your Google Cloud account for the resources used in this post, follow these steps:
- In the Google Cloud console, go to the Manage resources page
- In the project list, select the project that you want to delete, and then click Delete
- In the dialog, type the project ID, and then click Shut down to delete the project
- If you want to keep the project and just delete the Cloud Run service, follow the step here
- If you want to just delete the Cloud Function or revoke access, you can do so from here
7. Congratulations
Congratulations! You have successfully used a Vertex AI PaLM2 API to programmatically perform text summarization, build a Svelte web app and deploy in Cloud Functions. Check out Vertex AI LLM product documentation to learn more about available models.