Introduction to testing with Gemini Code Assist

1. Introduction

In this lab, you'll use Gemini Code Assist, an AI-powered collaborator in Google Cloud, to add tests to an existing Python web application, and find and fix errors in that application exposed by the tests. You will then use Code Assist to create tests for new functionality and generate code to pass those tests and extend the app.

What you'll do...

You'll use Cloud Shell Editor to download code for an existing Web application.
You'll use Gemini Code Assist Chat in Cloud Shell Editor to ask general questions about Google Cloud.
You'll use Gemini Code Assist Inline code assistance in Cloud Shell Editor to generate tests for the application, run the tests, and find and fix errors, and then to extend the application's functionality.

What you'll learn...

How to use Gemini Code Assist for several developer tasks like test generation and code generation.
How to use Gemini Code Assist to learn about Google Cloud.

What you'll need...

Chrome web browser
A Gmail account
A Cloud Project with billing enabled
Gemini Code Assist enabled for your Cloud Project

This lab is targeted to developers of all levels, including beginners. Although the sample application is in the Python language, you don't need to be familiar with Python programming in order to understand what's going on. Our focus will be on getting familiar with the capabilities of Gemini Code Assist for developers.

2. Setup

You should already have a Cloud Project with billing enabled to use for this lab. We will now enable Gemini API in our Google Cloud Project. Follow the steps given below:

Visit https://console.cloud.google.com and ensure that you have selected the Google Cloud Project that you plan to work with for this lab. Click on the Gemini icon that you see in the top right.

The Gemini for Cloud console window will open up on the right side of the console. Click on the Enable button if shown below. If you do not see the Enable button and instead see a Chat interface, you have already enabled Gemini for Cloud for the project and you can directly go to the next step.

Once it is enabled, you can test out Gemini by asking it a query or two. A few sample queries are shown but you can try something like What is Cloud Run?

Code Assist will respond with the answer to your question. You can click on the icon on the top right corner to close the Code Assist chat window.

Enable Gemini in Cloud Shell Editor

Gemini Code Assist is available and behaves similarly in several popular IDEs. You will be using Google Cloud Shell Editor, which runs completely in your web browser, in this codelab. You need to enable and configure Gemini in the Cloud Shell Editor and the steps are given below:

Launch Cloud Shell via the icon shown below. It may take a minute or two to start up the Cloud Shell instance.

Click on Editor or Open Editor button (as the case might be) and wait till the Cloud Shell Editor appears. If you see the Try the new editor button, click that.

Click on the Cloud Code - Sign in button in the bottom status bar as shown. Authorize the plugin as instructed. If you see "Cloud Code - no project" in the status bar, select that and then select the specific Google Cloud Project from the list of projects that you plan to work with.

If you do not see the Gemini icon in the status bar in the right bottom, you will need to enable it in Cloud Code. Before you do that ensure that Gemini (formerly known as Duet AI for Developers) is enabled in the IDE by going to the Cloud Code Extension → Settings and then enter the text Duet AI: Enable as shown below. Ensure that the checkbox is selected. You should reload your IDE. This enables Gemini in Cloud Code, and the Gemini status bar will appear in your IDE.

Click on the Gemini button in the bottom right corner as shown and select the correct Google Cloud project for which we had enabled the Cloud AI Companion API.

Once you select the your Google Cloud project, ensure that you are able to see that in the Cloud Code status message in the status bar and that you also have Gemini enabled on the right, in the status bar as shown below:

Gemini Code Assist is ready to use!

3. Download and examine the application

In the terminal window, run the command to clone the repository with the starting code and then change to the new directory (if the terminal window is no longer open, click the Terminal or Open Terminal button to restore it):

git clone https://github.com/GoogleCloudPlatform/testing-with-duet-ai-codelab.git
cd testing-with-duet-ai-codelab

Open main.py in the editor, then open the Gemini Chat window by clicking the Gemini chat icon on the left hand side of the editor. This Gemini Chat window is within the IDE and has the code in the IDE available as context for discussion. Enter the prompt Explain this and view the answer:

You can scroll this chat window to see the entire answer. The explanation says we can run this program locally with the command python3 main.py in the terminal window.

4. Run locally

Change to the repository directory with cd ~/testing-with-duet-ai-codelab if needed, and enter the command python3 main.py in the terminal window:

Click on the link http://127.0.0.1:8080 to open a new browser tab to the application's home page:

The application is running "locally". Actually, the Cloud Shell Editor has done a little bit of magic here. The application is running in Cloud Shell, not on your own computer. When you clicked the link, it opened a tab not to the actual local address http://127.0.0.1:8080, but to a proxy server set up just for this purpose by Cloud Shell. The effect is the same as if you were really running it locally.

Try it out. Enter 25 and press Convert!

That's right, 25 is XXV in Roman Numerals! You must be done here.

Maybe check a few more numbers. 25 worked, what about 24?

Perhaps we were a little hasty to think everything was okay. Is XXIIII the correct conversion for 24? Shouldn't it be XXIV?

A case could be made that XXIIII is right, but it's not really what people usually expect. Since it's not really wrong though (note that many clocks show 4 as Roman Numeral IIII) leave that issue for a future enhancement.

How about trying negative numbers? Zero? There's no way to represent those numbers in Roman Numerals. Nothing appears to come back to the user, which looks like an error needing to be addressed.

Testing can help find and eliminate errors, and Gemini Code Assist can help us write and use tests.

5. Adding Tests

Return to the Gemini Chat window, and ask

How can I test the number_to_roman function?

Read through the response, which should include discussing the unittest module and the pytest module.

You might like to have Gemini Code Assist actually write these tests for you. Open calendar.py, where the actual conversion code lies, in the editor, return to the Gemini Chat window, and again ask

How can I test the number_to_roman function?

The response is more specific now, even including a unittest module you can copy or inject into a new file:

import unittest
import calendar

class NumberToRomanTest(unittest.TestCase):

    def test_convert_1(self):
        self.assertEqual(calendar.number_to_roman(1), "I")

    def test_convert_4(self):
        self.assertEqual(calendar.number_to_roman(4), "IV")

    def test_convert_9(self):
        self.assertEqual(calendar.number_to_roman(9), "IX")

    def test_convert_40(self):
        self.assertEqual(calendar.number_to_roman(40), "XL")

    def test_convert_90(self):
        self.assertEqual(calendar.number_to_roman(90), "XC")

    def test_convert_400(self):
        self.assertEqual(calendar.number_to_roman(400), "CD")

    def test_convert_900(self):
        self.assertEqual(calendar.number_to_roman(900), "CM")

    def test_convert_1990(self):
        self.assertEqual(calendar.number_to_roman(1990), "MCMXC")

    def test_convert_2023(self):
        self.assertEqual(calendar.number_to_roman(2023), "MMXXIII")

You might see different code from the example above. The underlying models of Gemini Code Assist are updated from time to time, so the answers won't always be the same. If you see a different set of code, you can now choose whether to continue with samples shown in this codelab by copying the code shown here, or you can try out the alternate answer Gemini Code Assist is now giving you. If you have time, you can even try both paths. Gemini Code Assist is a coding assistant for you to use as you see fit.

Either click the double pointed arrow in the upper right corner of the Gemini Chat window to create a new file containing the unit test code, or use the IDE to create a new file and paste the code shown in this lab. Press CTRL-S or CMD-S in that window to save it, and call the saved file calendar-unittest.py.

Return to the terminal and press CTRL-C to stop the web server you left running earlier, and get a shell prompt. Enter the command

python3 calendar-unittest.py

to run the new tests.

There is no output. That's not what was expected. Did everything pass silently? You would like to know that for sure. Look back at the answer from Gemini Code Assist that included the test code. Below the code there was more information about how to run the test case:

Try running the recommended command:

python -m unittest discover

You may have an issue if your machine does not alias the python3 command to python, in which case run:

python3 -m unittest discover

The command runs, but it returns Ran 0 tests in 0.000s. The module has several tests in it. What's happening?

It's that last word in the command, discover. Where did it come from? Apparently, Gemini Code Assist expected the test code to be saved in a file named discover or discover.py, but didn't specify that's what you should do. Since you actually saved the file in calendar-unittest.py, try running the command:

python3 -m unittest calendar-unittest

Now you see a lot of output, starting with something like this:

$ python3 -m unittest calendar-unittest
.F.FFFFFF
======================================================================
FAIL: test_convert_1990 (calendar-unittest.NumberToRomanTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/charles_engelke/testing-with-duet-ai-codelab/calendar-unittest.py", line 28, in test_convert_1990
    self.assertEqual(calendar.number_to_roman(1990), "MCMXC")
AssertionError: 'MDCCCCLXXXX' != 'MCMXC'
- MDCCCCLXXXX
+ MCMXC

The first line shows a period for each passing test, and an F for each failing one. Most of the tests are failing! It then lists the failing tests individually, showing the expected output and the actual output. It's a bit unclear in what order these tests were run. It was in alphabetical order by test name, not the order the tests appear in the file. So test_convert_1 ran first, then test_convert_1990, then test_convert_2023, and so on. The test cases for 1 and 2023 are the only ones that passed.

When you first tried out this code you noticed that it converted 24 to XXIIII, which was not exactly wrong, but not the common form where IIII is converted to IV. All the failing tests were for similar cases. When this problem was first noted, the lab said, "Since it's not really wrong though (note that many clocks show 4 as Roman Numeral IIII) leave that issue for a future enhancement."

You could change the test cases to expect and accept the "not really wrong" answers the code is given, or accept that it's time for that "future enhancement." So your next step is to fix the code, with Gemini Code Assist help, to give the more acceptable answers the tests expect.

6. Enhancing the code

Recall that responses like XXIIII for 24, instead of the more common XXIV, were deemed "not really wrong" and were put off for a future enhancement. That future is now. Those "not really wrong" answers are still annoying.

The first rule for repeated digits in Roman Numerals is: any time you have four identical digits in a row, they should be replaced by one of the digits followed by the next higher digit. So XXIIII should be replaced by XXIV. Similarly XXXX should be changed to XL, and CCCC should become CD.

Ask Gemini Code Assist how to change the value of the roman variable in this way just before it is returned by number_to_roman:

If the final value of roman has IIII in it, that should be replaced by IV. Similarly XXXX should be replaced by XL, and CCCC should become CD. How can I make those changes?

The suggestion is to add some code at the end:

Copy/paste or type those lines in the editor, and then look at what happens:

Gemini Code Assist has added more lines to handle the cases you can end up with after making the first set of substitutions. For example, 19 would be converted to XVIIII, then to XVIV, and finally to the correct XIX.

If Gemini Code Assist made apparently useful suggestions, press Tab to accept the recommendations, save the file, and run the web server again. Otherwise, add the lines shown in the example here manually, and save the file. Try a tough conversion: 1999:

That's correct!

Rerun the tests now. They all pass!

The web application seems ready to put into production.

7. Deploy to Cloud Run

Cloud Run will run a containerized application on the Internet for you. For applications written using common frameworks, like Flash, the gcloud run deploy command will even build that container for you before deploying it. Run the command:

gcloud run deploy

In the terminal. When asked the location of the source code, press Enter to accept the correct location it suggests. Similarly, when asked for a service name, press Enter to accept the suggestion.

The command may fail because gcloud cannot determine which project to use. In that case, run the command:

gcloud config set core/project <project-id>

where is replaced by your project's id, which may be the same as its name. Then rerun the gcloud run deploy command.

The command will prompt you that certain APIs are needed and not yet enabled. Enter y to have them enabled for you.
When asked to select a region, pick one convenient to you. Entering the number corresponding to us-central1 is a safe choice.
When asked, enter Y to continue.
You will want to allow unauthenticated invocations of this Cloud Run service. The authentication option used by Cloud Run is suitable for use by programs calling the service. Since this is a website, you will not use authentication.

Google Cloud will build the container, deploy it, route traffic to it, and set access policies, and then show you the link to the home page:

You can go to that link and access your application.

Enter a number and press Enter, and tada!

What!?!

It worked on your machine! Why isn't this finished?

Find out. Ask Gemini Code Assist,

Why am I getting an internal server error on cloud run?

Apparently, Gemini Code Assist can read the log file, which says something similar. Let's ask Gemini Code Assist how you can look at to logs yourself:

Go ahead and do that. Look for lines with red !! indicators of error, as below:

This is followed by many lines of detail on the call stack getting here, but then there is this:

When you look at your calendar.py file, you see the function number_to_roman right there! And you know it's right because it worked on your machine. What could be different in Cloud Run?

The answer is tricky. There is a standard module included with Python3 called calendar, just like the calendar.py file the number_to_roman function is defined in. On your local machine, when Python looked for a module called calendar, it searched your application directory first. Apparently, Python on Cloud Run looked for standard modules first, imported it, and did not find a number_to_roman function.

These kinds of differences in environments are always possible. Fortunately, when an application is containerized it carries its environment within it, so wherever you run it you can expect the same behavior. If you had run the same containerized application locally that Cloud Run has, you'd have had the same problem.

Fix this problem. You need to change the name of your local calendar module to something that isn't also a standard module name. Rename the calendar.py file to my_calendar.py, then change the import calendar lines in main.py and calendar-unittest.py to import my_calendar. Finally, change the line

roman = calendar.number_to_roman(number)

roman = my_calendar.number_to_roman(number)

Try it out locally, run the tests, and then redeploy:

gcloud run deploy

And now it works:

You can share this URL, and everybody that needs a Roman Numeral conversion tool can use yours.

8. Optional: Make it look nicer

Your application is running fine, and accessible to anybody on the web. But it is a little plain looking. Before you tell everybody about it, why not ask Gemini Code Assist to improve its appearance?

Open the templates/index.html file. In the Gemini chat window, ask:

Make this index.html file use material design.

The response is to make additions to the current file, resulting in something similar to the following:

<!DOCTYPE html>
<html>
<head>
    <title>Roman Numerals</title>
    <link rel="stylesheet" href="https://code.getmdl.io/1.3.0/material.indigo-pink.min.css">   
    <script defer src="https://code.getmdl.io/1.3.0/material.min.js"></script>     
</head>
<body>
    <h1 class="mdl-typography--title">Roman Numerals</h1>
    <form action="/convert" method="post">
        <div class="mdl-textfield mdl-js-textfield">
            <input class="mdl-textfield__input" type="text" id="number" name="number" required />
            <label class="mdl-textfield__label" for="number">Enter a number:</label>
          </div>
          <button class="mdl-button mdl-js-button mdl-button--raised mdl-button--colored">
            Convert!
          </button>
    </form>
</body>
</html>

Use the icon to copy the suggested code and paste over the existing contents of index.html. In the terminal, run python3 main.py and click the link to open a preview window. The page is a little bit less plain now:

You can repeat this with the convert.html file if you wish.

Gemini Code Assist knows quite a bit of CSS, and you can have it help you style the application pages in a variety of ways. This is just a start.

Since you want to share this application, don't forget to redeploy it to Cloud Run:

gcloud run deploy

You can pass on the URL to people who need to convert to Roman Numerals.

9. Congratulations!

Congratulations - you've successfully worked with Gemini Code Assist to add tests to an application, fix errors in it, and add enhanced functionality.

When you are finished using the application you build, you can delete it from the cloud console dashboard to stop any potential future charges.