Writing Synthetic Monitoring Tests for your services using Gemini

1. Introduction

In this codelab, we shall look at utilizing the Help Me Write feature to author Synthetic Monitoring Tests for your existing services.

What you'll do...

You will deploy an API to Google Cloud Run, which will act as a basis for our service that we shall test.
You will then write a Synthetic Monitor, which is a feature of Cloud Monitoring.
You will use the Help Me Write feature to write the Synthetic Monitor.

What you'll learn...

What Synthetic Monitoring is.
How to use Gemini's Help me Write feature in Synthetic Monitoring, to author test cases that will validate the core service functionality.

What you'll need...

Chrome web browser
A Gmail account
A Cloud Project with billing enabled
Gemini Code Assist enabled for your Cloud Project

This lab is targeted to developers of all levels, including beginners. Although the sample application is in Python language, you don't need to be familiar with Python programming in order to understand what's going on.

2. Setup

We will now enable Gemini for Cloud in our Google Cloud Project. Follow the steps given below:

Visit https://console.cloud.google.com and ensure that you have selected the Google Cloud Project that you plan to work with for this lab. Click on the Open Gemini icon that you see in the top right.

Gemini for Cloud chat window will open up on the right side of the console. Click on the Enable button as shown below. If you do not see the Enable button and instead see a Chat interface, it is likely that you have already enabled Gemini for Cloud for the project and you can directly go to the next step.

Once it is enabled, you can test out Gemini for Cloud by asking it a query or two. A few sample queries are shown but you can try the following query What is Synthetic Monitoring?

Gemini for Cloud will respond with the answer to your question. Check out the list of reference Documentation that has been provided on how you can write Synthetic Monitors in Google Cloud.

You can click on the icon on the top right corner to close the Gemini for Cloud chat window.

3. Deploy a sample Inventory API on Google Cloud Run

Before we get writing with the tests, we will need a sample API that we can test against. For this, we will be writing a simple inventory API that we will deploy on Google Cloud Run.

We will be using Cloud Shell IDE, a fully-managed Code OSS-based development environment. The environment comes with Cloud Code IDE Extension that makes working with Google Cloud services efficient. Follow the steps given below:

Visit ide.cloud.google.com. It may take a while for the IDE to appear, so please be patient.
Click on the Cloud Code - Sign in button in the bottom status bar as shown. Authorize the plugin as instructed. If you see "Cloud Code - no project" in the status bar, select that and then select the specific Google Cloud Project from the list of projects that you plan to work with.

Click on the Gemini button in the bottom right corner as shown and select one last time the correct Google Cloud project. If you are asked to enable the Cloud AI Companion API, please do so and move forward.
Once you've selected your Google Cloud project, ensure that you are able to see that in the Cloud Code status message in the status bar and that you also have Code Assist enabled on the right, in the status bar as shown below:

Click on the Google Cloud project name in the status bar below.

A list of options will appear. Click on New Application from the list below.

Select Cloud Run application.
Select the Python (Flask): Cloud Run application template.
Save the new application in your preferred location.
A notification confirms that your application was created, and a new window opens with your application loaded as shown below. A README.md file is opened. You can close that view for now.

From the Explorer, visit app.py file and replace it with the contents shown below:

from flask import Flask, jsonify, request

app = Flask(__name__)

inventory = [
    {
        'id': 1,
        'name': 'Item 1',
        'quantity': 10
    },
    {
        'id': 2,
        'name': 'Item 2',
        'quantity': 20
    },
    {
        'id': 3,
        'name': 'Item 3',
        'quantity': 30
    }
]

@app.route('/inventory', methods=['GET'])
def get_inventory():
    return jsonify(inventory)

@app.route('/inventory/<int:id>', methods=['GET'])
def get_inventory_item(id):
    for item in inventory:
        if item['id'] == id:
            return jsonify(item)
    return jsonify({'error': 'Item not found'}), 404

if __name__ == '__main__':
    app.run(debug=True, host='0.0.0.0', port=8080)

It's time to deploy the inventory API to Cloud Run. We will need to launch a new Terminal session from Cloud Shell IDE via the menu icon in the top left. Click on the menu icon and then Terminal → New Terminal as shown below:

In the Terminal session, give the following command:

gcloud run deploy --source .

The above command will ask about a region to deploy, please choose us-central1. It will ask for allowing unauthenticated invocations, please allow that by saying y.
Once the service is deployed successfully, it will provide a Service URL. Note that down.

4. Test out the Inventory API

You can now test out the Inventory API, by launching the browser and hitting the following endpoints:

SERVICE_URL/inventory

This should return you 3 inventory items as per the sample data in the service. The sample response is given below:

[
  {
    "id": 1,
    "name": "Item 1",
    "quantity": 10
  },
  {
    "id": 2,
    "name": "Item 2",
    "quantity": 20
  },
  {
    "id": 3,
    "name": "Item 3",
    "quantity": 30
  }
]

We can now retrieve a specific inventory item via the URL below. This should return you the inventory item that has the id value of 1.

SERVICE_URL/inventory/1

The response should be similar to:

{
  "id": 1,
  "name": "Item 1",
  "quantity": 10
}

Finally, we can try to retrieve an inventory item that does not exist.

SERVICE_URL/inventory/200

This should return you an error message since there is no inventory item with the id value of 200. The response should be similar to the following:

{
  "error": "Item not found"
}

We are now ready to write our Synthetic Monitoring Tests in Cloud Monitoring with the help of Gemini.

5. Synthetic Monitoring in Google Cloud

As the documentation states, Synthetic monitors let you define what you are going to test and a sequence of tests. For example, you can test the login page of your application, the checkout process of your ecommerce store, or the API calls that your application makes to third-party services.

When you create a synthetic monitor, you deploy a 2nd gen Cloud Function which is built on Cloud Run. Your function must be written in Node.js and rely on the open source Synthetics SDK framework. Cloud Monitoring distributes and manages this framework.

Cloud Monitoring supports the following types of synthetic monitors:

Custom or Mocha-based synthetic monitors let you deploy a fully-configurable single-purpose Cloud Function.
Broken-link checkers let you specify options, such as the origin URI, the number of links tested, and the number of retries, before deploying a preconfigured Cloud Function.

Cloud Monitoring does a lot of heavy lifting while executing these synthetic monitors. It is responsible for:

Periodic execution of your Cloud Function.
Collecting and storing the results of each execution:
Success and failure information, such as the error message, error type, and line of code
Execution time
Logs
Metrics

We are going to take the help of Gemini and specifically the Help Me Write feature that will provide us with the initial code for the tests, so that we can use this to test out and also build out additional features on top of it. Let's get going.

6. Writing our Synthetic Monitor for the Inventory API Test Cases

We are now going to the Cloud Console to write our Synthetic Monitor.

Visit the Synthetic Monitoring page in Cloud Console. This will bring up the page as shown below:

Click on the CREATE SYNTHETIC MONITOR link in the above page. This will bring up a configuration form as shown below:

We have provided a name s1, but you can choose any other name. Notice the HELP ME CODE button in the screen above. Click on that.

This will bring up a popup where you will need to give the prompt that describes the test cases that you want to execute to ensure that the Inventory API is working fine.

In the prompt edit box, use a prompt like the one given below:

Help me write Node.js based synthetic test script with the assert module that performs the following tests:
1. A GET operation on SERVICE_URL/inventory. The http response code returned should be 200. 
2. A GET operation on SERVICE_URL/inventory/1. The http response code returned should be 200. The response data is JSON format. It should be a single object with attributes as follows: "id" value should be 1, "name" value should be "Item 1" and "quantity" value should be 10.
3. A GET operation on SERVICE_URL/inventory/200. The https response code returned should be 404. The response data is in JSON format. It should be a single object with attributes as follows: "error" value should be "Item not found".

Notice that we have provided 3 test cases and you should replace the SERVICE_URL value with the actual Cloud Run Service name for your service.

Once you click on GENERATE, Gemini will generate the Test Suite code and the package.json file too as shown below. The code generated for you might differ from the one that you see below. Take a look at the code and the dependency files in the package.json file.

Click on INSERT INTO CLOUD FUNCTION. This will bring up a Cloud Function creation form with the necessary values populated for you.

Click on APPLY FUNCTION and then the CREATE button. We did not choose any Alert Channel configuration in this example but you are free to select that.

This will start the process of creating the Google Cloud Function in the background. This could take a few minutes, so be patient.

Once the Cloud Function has been successfully deployed, Cloud Monitoring will begin the task of invoking the Synthetic Monitor for you.

At the beginning you will see that there are no invocations, as shown below:

Once there are specific runs, you will be able to see the various executions. The screen below demonstrates that the tests are passing:

If you click on the Synthetic Monitor name (e.g. s1) it will show the various executions as shown below:

7. Congratulations!

Congratulations - you've successfully deployed a sample API to Google Cloud Run and wrote Synthetic Monitoring tests to validate the functionality of the service. During the process, you utilized Gemini to help generate the code for the Test Suite.