Using Gemini Code Assist to explore and enhance AI Summarization Jump Start Solution

1. Introduction

In this codelab, we shall look at an existing Jump Start Solution, AI Summarization, that uses Vertex AI models to summarize PDF documents that have been uploaded to Google Cloud Storage.

We will then use Gemini Code Assist to:

• Understand the Python code that powers the Cloud Function that does the work of extracting the text from the PDF document, summarizing it and writing the results to BigQuery.
• We shall take the help of Gemini Code Assist throughout the process to help write new functionality. We will develop a Web application (Python Flask Application) and run the application locally to verify our code.
• Optionally, we can also look at deploying this application to Cloud Run and improve the design (aesthetics) of the web application using Material Design.

What you'll do...

• You will deploy the AI Summarization Jump Start Solution and trigger the process flow to understand how it works.
• You will then use Cloud Shell IDE to download the existing code for the Jump Start Solution and use Gemini Code Assist to understand the code.
• You'll use Gemini Code Assist Cloud Shell IDE to generate code for a new functionality.

What you'll learn...

• How AI Summarization Jump Start Solution works.
• How to use Gemini Code Assist for several developer tasks like code generation, code completion and code summarization.

What you'll need...

• Chrome web browser
• A Gmail account
• A Cloud Project with billing enabled
• Gemini Code Assist enabled for your Cloud Project

This lab is targeted to developers of all levels, including beginners. Although the sample application is in Python language, you don't need to be familiar with Python programming in order to understand what's going on. Our focus will be on getting familiar with the capabilities of Gemini Code Assist for developers.

2. Setup

This section covers everything you need to do to get started with this lab.

Enable Gemini for Cloud in the Google Cloud Project

We will now enable Gemini for Cloud in our Google Cloud Project. Follow the steps given below:

1. Visit https://console.cloud.google.com and ensure that you have selected the Google Cloud Project that you plan to work with for this lab. Click on the Open Gemini icon that you see in the top right.

2. Gemini for Cloud chat window will open up on the right side of the console. Click on the Enable button as shown below. If you do not see the Enable button and instead see a Chat interface, it is likely that you have already enabled Gemini for Cloud for the project and you can directly go to the next step.

3. Once it is enabled, you can test out Gemini for Cloud by asking it a query or two. A few sample queries are shown but you can try something like What is Cloud Run?

Gemini for Cloud will respond with the answer to your question. You can click on the icon on the top right corner to close the Gemini for Cloud chat window.

Enable Gemini Code Assist in Cloud Shell IDE

We will be using Cloud Shell IDE, a fully-managed Code OSS-based development environment, for the rest of the codelab. We need to enable and configure Code Assist in the Cloud Shell IDE and the steps are given below:

1. Visit ide.cloud.google.com. It may take a while for the IDE to appear, so please be patient.
2. Click on the Cloud Code - Sign in button in the bottom status bar as shown. Authorize the plugin as instructed. If you see "Cloud Code - no project" in the status bar, select that and then select the specific Google Cloud Project from the list of projects that you plan to work with.

3. Click on the Gemini button in the bottom right corner as shown and select one last time the correct Google Cloud project. If you are asked to enable the Cloud AI Companion API, please do so and move forward.
4. Once you've selected your Google Cloud project, ensure that you are able to see that in the Cloud Code status message in the status bar and that you also have Code Assist enabled on the right, in the status bar as shown below:

Gemini Code Assist is ready to use!

Gemini Code Assist is also available in local IDEs such as Visual Studio Code, or JetBrains IntelliJ. It is also available in Google Cloud Workstations. This codelab focuses on Cloud Shell IDE as a way to get everyone to use the same environment but feel free to try your favorite setup and check out this link for more details.

Optional: If you do not see the Gemini in the status bar in the right bottom, you will need to enable Gemini in Cloud Code. Before you do that ensure that Gemini is enabled in the IDE by going to the Cloud Code Extension → Settings and then enter the text Gemini as shown below. Ensure that the checkbox is selected. You should reload your IDE.This enables Gemini in Cloud Code, and the Gemini icon in the status bar will appear in your IDE.

3. Deploy the AI Summarization Jump Start Solution

1. Go to the Generative AI document summarization solution
2. Click Deploy

• If your project does not have billing enabled, enable billing.
• Select us-central1 as the region.
• Click deploy.
• This can take up to 15 minutes.
• You do not need to make any changes, but feel free to explore the Jump Start Solution deployment by clicking on the EXPLORE THIS SOLUTION button in the solution deployment detail page.

4. Chat with Gemini

We're going to start by learning how to chat with Gemini. Gemini is available as a chat assistant within the Cloud Shell IDE as part of Cloud Code extension in VS Code. You can bring it up by clicking on the Gemini button in the left navigation bar. Look for the Gemini icon in the left navigation toolbar and click on that.

This will bring up the Chat: GeminiI pane inside Cloud Shell IDE and you can chat with Gemini to get help on Google Cloud.

Let us use the Gemini chat pane to enter a prompt and view the response from Gemini. Enter the following prompt:

What is Cloud Run? 

Gemini should respond with the details about Cloud Run. Prompts are questions or statements that describe the help that you need. Prompts can include context from existing code that Google Cloud analyzes to provide more useful or complete responses. For more information on writing prompts to generate good responses, see Write better prompts for Gemini in Google Cloud.

Try out the following sample prompts or any of your own to ask questions about Google Cloud:

• What is the difference between Cloud Run and Cloud Functions?
• What services are available on Google Cloud to run containerized workloads?
• What are the best practices to optimize costs while working with Google Cloud Storage?

Notice the trash can icon at the top - this is your way to reset the context for the Code Assist chat history. Note also that this chat interaction is contextual to the file(s) that you are working on in the IDE.

5. Download the Jump Start Solution Cloud Function in Cloud Code

Assuming that you are in Cloud Shell Editor, follow these steps:

• Click Cloud Code
• Note: Depending on the size of your screen, it may take one step or two steps.

or

• Click Cloud Functions.
• If prompted, log in or authorize your account.
• Click the webhook function.
• Click the Download to new workspace icon
• 
• Use webhook-1 as the workspace name (should be the default) or any other name and click OK.
• This will open up the code in the Cloud Shell IDE.

6. Review the existing project

This Jump Start solution is shown below:

Review the flow from the Upload PDF functionality to Cloud Storage. The Cloud Function that will get invoked if the PDF file is uploaded is given in the main.py file.

Click on that file. The entry point for the cloud function is entrypoint function, that eventually invokes the cloud_event_entrypoint function that extracts the text from the PDF, then invokes the summarization_entrypoint, that uses Vertex AI Models to summarize and write the results to GCS and BigQuery respectively.

Highlight all the code in the main.py file or any specific snippet of code. Click the Gemini Chat and give the following prompt: Explain this.

This should give you an explanation for the code.

7. Execute a sample run

As per the architecture diagram, we are going to upload a file into the <PROJECT_ID>_uploads bucket for the Cloud Function to get invoked.

Make sure that you have a sample PDF ready that you can upload and for which you would like the summarization.

• Go to Google Cloud Storage in the Cloud Console.
• Go to the <PROJECT_ID>_uploads bucket. Click on the UPLOAD FILES link and upload a sample PDF.

If you don't have a sample PDF file, you can use one of the samples we have created. From Cloud Shell, run the following command:

gsutil cp \
    gs://arxiv-dataset/arxiv/cmp-lg/pdf/9410/9410009v1.pdf \
    gs://<PROJECT_ID>_uploads/

Once the file is successfully uploaded, the webhook cloud function will get invoked and it will summarize the text found in the document. The output will be written to a BigQuery dataset named summary_dataset and in the summary_table table.

Query the table to see the results of the summary.

8. Make a web application client for the application

The above process is a manual step by step process to upload the PDF that we want summarized. How about building a web front-end to the application?

The requirements for the web front-end are simple:

1. A basic HTML form that allows us to select and upload the file, that needs to be summarized.
2. The file on successful upload, should be written to the <PROJECT_ID>_uploads bucket, so that the rest of the functionality works as is.

We will be using Python and the Flask framework for web applications to build this out in assistance with Duet AI.

Let's get started. We will assume that you still have the same workspace open in the Cloud Shell IDE.

Close all the files and give the give the following prompt in the Gemini Chat window:

Write a Python Flask application that has a couple of routes:
The root route / should serve the index.html page using the render_template framework. The /upload route should accept a file being uploaded and write that file to a Cloud Storage bucket.

Ideally this should produce code as follows:

from flask import Flask, render_template, request, redirect, url_for
from google.cloud import storage

app = Flask(__name__)

@app.route('/')
def index():
    return render_template('index.html')

@app.route('/upload', methods=['POST'])
def upload():
    file = request.files['file']
    filename = file.filename
    bucket = storage.Client().bucket('your-bucket-name')
    blob = bucket.blob(filename)
    blob.upload_from_string(
        file.read(),
        content_type=file.content_type
    )
    return redirect(url_for('index'))

if __name__ == '__main__':
    app.run(debug=True)

Save the above code as app.py in the root of the IDE workspace.

We would like the application to run on port 8080 and use host address 0.0.0.0 , so let's prompt Gemini to modify the app.run statement.

Give the following prompt:

Modify the app.py to run on port 8080 and host address 0.0.0.0

Your code should look like this now:

from flask import Flask, render_template, request, redirect, url_for
from google.cloud import storage

app = Flask(__name__)

@app.route('/')
def index():
    return render_template('index.html')

@app.route('/upload', methods=['POST'])
def upload():
    file = request.files['file']
    filename = file.filename
    bucket = storage.Client().bucket('your-bucket-name')
    blob = bucket.blob(filename)
    blob.upload_from_string(
        file.read(),
        content_type=file.content_type
    )
    return redirect(url_for('index'))

if __name__ == '__main__':
    app.run(debug=True, host='0.0.0.0', port=8080)

Next up, we would like the index.html to be generated too. Give the following prompt in Gemini Chat window:

Provide the index.html file

It should provide you with code as given below for index.html.

<!DOCTYPE html>
<html>
<head>
  <title>File Upload</title>
</head>
<body>
  <h1>File Upload</h1>
  <form action="/upload" method="post" enctype="multipart/form-data">
    <input type="file" name="file">
    <input type="submit" value="Upload">
  </form>
</body>
</html>

We can ask Gemini to tell us where to save the index.html file. Give the following prompt:

Since this is using the render_template framework, where should this file be saved?

It should give an answer that is similar to the following:

Go ahead and create a folder named templates in the root folder and save the index.html file in that.

We would like some exception handling in the app.py file in case the file could not get saved in the Google Cloud Storage bucket. Keep the app.py file open and give the following prompt in the Gemini Chat window.

modify the app.py to have exception handling in the upload() function. The success or failure should be returned to the index page.

This should now add some exception handling as shown below:

from flask import Flask, render_template, request, redirect, url_for
from google.cloud import storage

app = Flask(__name__)

@app.route('/')
def index():
    return render_template('index.html')

@app.route('/upload', methods=['POST'])
def upload():
    try:
        file = request.files['file']
        filename = file.filename
        bucket = storage.Client().bucket('your-bucket-name')
        blob = bucket.blob(filename)
        blob.upload_from_string(
            file.read(),
            content_type=file.content_type
        )
        return redirect(url_for('index'))
    except Exception as e:
        return render_template('index.html', error=str(e))

if __name__ == '__main__':
    app.run(debug=True, host='0.0.0.0', port=8080)

Since we need to display the error message in the index.html, we need to modify that too. Let's ask Gemini to do that via the prompt below:

update the index.html to display the error message

This should give an updated index.html as shown below:

<!DOCTYPE html>
<html>
<head>
  <title>File Upload</title>
</head>
<body>
  <h1>File Upload</h1>
  {% if error %}
    <p style="color: red;">{{ error }}</p>
  {% endif %}
  <form action="/upload" method="post" enctype="multipart/form-data">
    <input type="file" name="file">
    <input type="submit" value="Upload">
  </form>
</body>
</html>

Ensure that at each of the above steps, you are saving the changes in the app.py and index.html file respectively.

The app.py does not have the correct bucket name and hence we can give that information to Gemini and ask it to do the changes. Additionally, we have to provide the project id for the storage.Client() instance. Hence give the following couple of prompts (replace <PROJECT_ID> with your Google Cloud Project ID) in Gemini Chat window and incorporate the changes:

Prompt 1

My bucket name is gemini-for-devs-demo_uploads, please change the code to use that.

Prompt 2

My project id is gemini-for-devs-demo, please change the storage.Client() to use that.

The final app.py file looks like this (my project id is shown below, but it should ideally be the one that you are working with and which you have given in the prompt above):

from flask import Flask, render_template, request, redirect, url_for
from google.cloud import storage

app = Flask(__name__)

@app.route('/')
def index():
    return render_template('index.html')

@app.route('/upload', methods=['POST'])
def upload():
    try:
        file = request.files['file']
        filename = file.filename
        bucket = storage.Client(project='gcp-experiments-349209').bucket('gcp-experiments-349209_uploads')
        blob = bucket.blob(filename)
        blob.upload_from_string(
            file.read(),
            content_type=file.content_type
        )
        return redirect(url_for('index'))
    except Exception as e:
        return render_template('index.html', error=str(e))

if __name__ == '__main__':
    app.run(debug=True, host='0.0.0.0', port=8080)

9. Run the web application locally

Create a Python environment with the dependencies defined in the requirements.txt file. Go to the Command Palette in Cloud Shell IDE as shown below:

Type in Python: Create Environment and then go through the steps to create a Virtual Environment using (venv), then Python 3.x interpreter and the requirements.txt file. This will create the required environment.

Launch the Terminal now, as shown below:

Give the following command in the terminal:

python app.py

The Flask app should launch and you should see something like this:

(.venv) romin@cloudshell:~/webhook-2 (gcp-experiments-349209)$ python app.py 
 * Serving Flask app 'app'
 * Debug mode: on
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:8080
 * Running on http://10.88.0.3:8080
Press CTRL+C to quit
 * Restarting with watchdog (inotify)
 * Debugger is active!
 * Debugger PIN: 989-296-833

Visit the http://127.0.0.1:8080 URL and it should display the index.html page

Upload a file from your local machine and it should be processed successfully.

You can check the summarization by going to the BigQuery dataset and table that we saw earlier in the lab. Alternatively, you can check out the Cloud Storage bucket (<PROJECT_ID>_output).

10. (Optional) Open Exploration - Deploy to Cloud Run

• You can deploy the application to Cloud Run.
• Ask Gemini Code Assist with the following prompt (Might need to try a few variations of the above prompt):

I don't want to build a container image but deploy directly from source. What is the gcloud command for that?

11. (Optional) Open Exploration - Add CSS Styles

• Use Gemini Code Assist and the in-editor assistant to add CSS styles to your application and deploy the application again when you're done!
• Open the index.html file and give the following prompt in the Gemini Chat: Can you apply material design styles to this index.html?
• Check out the code and see if it works.

12. Congratulations!

Congratulations - you've successfully worked with Gemini Code Assist on a sample project to understand how it can assist with Code Generation, Code Completion, Code Summarization and help you with getting answers to questions about Google Cloud.

13. Reference docs

• Gemini Code Asssit Home page
• Use Gemini for AI Assistance and Development - Documentation
• Gemini use cases