1. Overview
In today's fast-paced retail landscape, delivering exceptional customer service while enabling personalized shopping experiences is paramount. We'll take you on a technical journey through the creation of a knowledge-driven chat application designed to answer customer questions, guide product discovery, and tailor search results. This innovative solution combines the power of AlloyDB for data storage, an in-house analytics engine for contextual understanding, Gemini (Large Language Model) for relevance validation, and Google's Agent Builder for quickly bootstrapping an intelligent conversational assistant.
The Challenge: Modern retail customers expect instant answers and product recommendations that align with their unique preferences. Traditional search methods often fall short of providing this level of personalization.
The Solution: Our knowledge-driven chat application tackles this challenge head-on. It leverages a rich knowledge base derived from your retail data to understand customer intent, respond intelligently, and deliver hyper-relevant results.
What you'll build
As part of this lab (Part 2), you will:
- Build a Vertex AI Agent Builder agent
- Integrate the AlloyDB tool with the agent
Requirements
2. Architecture
Data Flow: Let's take a closer look at how data moves through our system:
Ingestion:
Retail data (inventory, product descriptions, customer interactions) is continuously loaded into AlloyDB.
Analytics Engine:
We will use AlloyDB as the analytics engine to perform the below:
- Context Extraction: The engine analyzes the data stored within AlloyDB to understand relationships between products, categories, customer behavior, etc as applicable.
- Embedding Creation: Embeddings (mathematical representations of text) are generated for both the user's query and the information stored in AlloyDB.
- Vector Search: The engine performs a similarity search, comparing the query embedding to the embeddings of product descriptions, reviews, and other relevant data. This identifies the 25 most relevant "nearest neighbors."
Gemini Validation:
These potential responses are sent to Gemini for assessment. Gemini determines if they are truly relevant and safe to share with the user.
Response Generation:
The validated responses are structured into a JSON array and the whole engine is packaged into a serverless Cloud Run Function that is invoked from the Agent Builder.
The above steps are already covered in part 1 of the lab.
We discussed the technical details of creating a knowledge-driven analytics engine that powers our smart shopping assistant. Now, let's explore how we leverage the magic of Agent Builder to bring this engine to life in a conversational interface. Make sure you have the endpoint URL ready before starting part 2. This next step is what we cover in this lab:
Conversational Interaction:
Agent Builder presents the responses to the user in a natural language format, facilitating a back-and-forth dialogue.
3. Before you begin
Create a project
- In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.
- Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project .
- You'll use Cloud Shell, a command-line environment running in Google Cloud that comes preloaded with bq. Click Activate Cloud Shell at the top of the Google Cloud console.
- Once connected to Cloud Shell, you check that you're already authenticated and that the project is set to your project ID using the following command:
gcloud auth list
- Run the following command in Cloud Shell to confirm that the gcloud command knows about your project.
gcloud config list project
- If your project is not set, use the following command to set it:
gcloud config set project <YOUR_PROJECT_ID>
- Enable the required APIs. The alternative to the gcloud command is through the console by searching for each product or using this link.
If any API is missed, you can always enable it during the course of the implementation.
Refer documentation for gcloud commands and usage.
Important Note: Also, make sure you have completed PART 1 of the lab in order to complete this.
4. Agent Creation
Introducing Agent Builder
Agent Builder is a powerful, low-code tool that empowers us to create conversational agents quickly and efficiently. It streamlines the process of designing dialog flows, integrating knowledge bases, and connecting to external APIs. In our case, we'll use Agent Builder to seamlessly connect with the Cloud Function endpoint we built in Part 1, enabling our shopping assistant to access our retail knowledge base and respond to customer queries intelligently.
Building the agent
Let's get started with creating this new agent for answering user questions for the apparel products.
- Start by logging into the Agent Builder platform. If it prompts you to activate the API, go ahead and click CONTINUE AND ACTIVATE THE API.
- Click on "CREATE APP" and give your agent a descriptive name (e.g., "Retail Shopping Assistant").
- Click App Type "Agent".
- . Give your agent a descriptive name like "Retail Shopping Assistant" and set the region as us-central1
- Enter the details for the agent:
- Change the Agent Name to "Retail Shopping Agent".
- Add the below "Goal":
You are a shopping assistant agent! Your job is to help the customer search for their ideal apparels, allow them to add items to their cart, remove items from their cart, and review items in their cart. Once a user is done searching, open the search results in a user friendly html page.
- Save it at this point and leave instructions blank for now.
- Then click Tools from the navigation menu and click CREATE.
Enter Tool Name: Retail Shopping Tool
Enter Tool Description:
This tool refers to the dataset in the backend as the context information for product inventory. It takes as input the user's search text summarized by the agent and matches with the most appropriate list of items and returns as an array of items.
Enter Schema — OpenAPI in YAML format:
This is the part where we are using the backend endpoint to power the agent. Copy the below OpenAPI spec and replace the URL placeholder (enclosed in angle brackets) with your Cloud Function endpoint:
openapi: 3.0.0
info:
title: AlloyDB Product Matcher
description: A Cloud Function to query AlloyDB for product matches based on user search text.
version: 1.0.0
servers:
- url: <<https://us-central1-YOUR_PROJECT_ID.cloudfunctions.net/alloy-gem>>
paths:
/:
post:
summary: Find matching products based on search text.
operationId: apparelSearch
requestBody:
description: JSON object containing the search text.
required: true
content:
application/json:
schema:
type: object
properties:
search:
type: string
description: The user's search query for product matching.
responses:
'200':
description: Successful response with a JSON array of matching products.
content:
application/json:
schema:
type: array
items:
type: object
properties:
id:
type: string
description: Product ID.
category:
type: string
description: Product category.
sub_category:
type: string
description: Product sub-category.
uri:
type: string
description: Product URI or URL.
description:
type: string
description: Product description.
literature:
type: object
description: JSON object containing match information from the ML model.
properties:
MATCH:
type: string
description: Whether the product matches the search query (YES/NO).
PERCENTAGE:
type: string
description: Percentage of match.
DIFFERENCE:
type: string
description: Description of differences between the search and product.
'500':
description: Internal server error.
Leave other configurations to their default values and click "Save".
- Go back to the Agent at this point because we want to add the "Tool" configuration to the agent's "Instructions". Add the below to the instructions placeholder (Remember, the indents are important in defining the flow):
- Greet the user and answer their question to the best of your knowledge.
- Summarize the user's request and ask them to confirm that you understood correctly.
- Check if the request has details like gender, color, material, style and other key apparel details already.
- If not, seek clarifying details.
- If the search request is very broad, then ask the user to narrow down the request with specific details that you believe could be personal preferences.
- Once all the necessary details are met, summarize your final understanding of the request with the user.
- Use ${TOOL: Retail Shopping Tool} to help the user with their task.
- If the request has unrelated input text, gracefully convey that you don't have information on that topic.
- Do not give product availability information outside the source that is provided to you in ${TOOL: Retail Shopping Tool}.
- Do not assist with any information unless you are certain that you know the answer and it is grounded in the source of truth.
- Thank the user for their business and say goodbye.
Make sure the tool "Retail Shopping Tool" is selected in the "Available Tools" section and then save the agent again.
5. Test the Agent
On the right pane, you should see the Preview Agent section that allows you to test your agent.
As you can see in the screenshot below, I have greeted as a user and started my chat with a request for white shirt with blue prints:
This is the JSON response:
This is the raw JSON result from the Cloud Function that processes the AlloyDB Similarity Search. That's it! We are all set with the agent now.
6. Sample Conversation Flow
At this point, you can test and iterate the flow of your dialog (conversation) agent:
User: I'm looking for women's boots for winter.
Agent: (OpenAPI call to Cloud Function that works with AlloyDB Similarity Search)
Cloud Function: (Processes query, returns JSON array)
Agent: Based on your preferences, here are some options: [Product 1, Product 2, Product 3]. Would you like to see more details about any of these?
User: No, That is all. Bye!
Agent: Have a good day!
Few Shot Prompts
Now let's say you want to see the result in the conversational agent in a certain graphic format including pictures and links. We will do that using examples of conversations also known as Few Shot Prompting.
This means we want to add a few examples to the agent builder to have a consistent result format.
In one of our previous screenshots in the "Test the Agent" section, we see the response from the agent as "I see. You are looking for a white shirt with blue prints...". Go to that response or create a new test conversation from the right pane of the agent:
- Type the below in the "Enter User Input" section:
You will see a response like this:
And you will also see the JSON Array returned by the API endpoint.
- Now click the Agent Name (as shown in the image below highlighted with a red tick mark) from the preview section's margin and all your tabs on the chat preview section will be highlighted. Now click the "Save Example" on the top right corner in the preview section. It will look like this:
- Save the conversation with a Display Name "Response with images" or something similar and click "Create".
- Now go to the response tab highlighted with a black tick mark in the above image and replace the text "I see. You are looking for yellow sandals. Is that correct?" with the following:
I see you are looking for yellow sandals. Here are a few options for you:
<!DOCTYPE html>
<html>
<body>
<h2>Featured Sandals</h2>
<table style="overflow-x: auto; white-space: nowrap;">
<tr>
<td>
<img src="https://assets.myntassets.com/v1/images/style/properties/d27dbd8e9666b9af2d72fbfde315f76d_images.jpg" alt="Yellow sandals from Estd. 1977" width="300">
</td>
<td>
<img src="https://assets.myntassets.com/v1/images/style/properties/b7a479fe5f56457e930541a789c5df68_images.jpg" alt="Yellow sandals from Gliders" width="300">
</td>
<td>
<img src="https://assets.myntassets.com/v1/images/style/properties/b6c813734b29b89d1055fd000ea50743_images.jpg" alt="Yellow sandals from Rocia" width="300">
</td>
<td>
<img src="https://assets.myntassets.com/v1/images/style/properties/ee0e918c229e76e0e7e61712e9d2ecb3_images.jpg" alt="Yellow flip flops from Numero Uno" width="300">
</td>
<td>
<img src="https://assets.myntassets.com/v1/images/style/properties/5bdd4c9e739205e28ee134ff7849bc60_images.jpg" alt="Yellow flip flops from Numero Uno" width="300">
</td>
</tr>
</table>
</body>
</html>
- Click "Save" on top.
Repeat the process for as many examples as you want with different variations and flows of the conversation.
Now go ahead and test it out with a user input:
Some other types of responses:
That's it. We have successfully created and tested a Conversational Agent for our retail app.
7. Deployment and Integration
Once you're satisfied with your agent, you can easily deploy it to various channels using Agent Builder's integrations. You can embed it on your website, integrate it with popular messaging platforms, or even create a dedicated mobile app. We can also use the Agent Builder API directly in our web client applications, which we have covered in this blog.
8. Clean up
To avoid incurring charges to your Google Cloud account for the resources used in this post, follow these steps:
9. Congratulations
Congratulations! By integrating the power of our custom-built analytics engine with the intuitive interface of Agent Builder, we've created a smart retail shopping assistant that delivers personalized experiences, answers questions accurately, and ultimately drives customer satisfaction and sales. By combining the capabilities of AlloyDB, Vertex AI, and Vector Search, we've taken a giant leap forward in making contextual and vector searches accessible, efficient, truly meaning-driven and agentic!