Looker Dashboard Summarization Extension Codelab

Looker Dashboard Summarization Extension Codelab

About this codelab

subjectLast updated Jun 8, 2024
account_circleWritten by Jeremy Chang

1. Before you begin

In this codelab (1) you will set up the Looker Dashboard Summarization Extension locally for you to try out and develop locally. Then (2) you will deploy the extension to production so other Looker users in your Looker instance can use it. Lastly, (3) you can follow additional steps to finetune and enhance the extension's functionality. All non-optional sections should be completed in sequential order.

Looker Dashboard Summarization Extension overview

Functionally, the Looker Dashboard Summarization Extension sends your Looker dashboard's data to Vertex AI's Gemini model. The Gemini model then returns a summary of your dashboard's data and a prescription of next steps. The extension displays the summary and next steps as a tile in your dashboard, integrating into your dashboard experience. Additionally, the extension can export the summary and next steps to Slack or Google Chat. The extension makes use of a React frontend application coupled with a websocket backend service to send and receive data to and from Vertex AI's Gemini model.

Prerequisites

  • Basic familiarity with Node development, Docker, and Terraform
  • Familiarity with setting up a Looker LookML project

What you'll learn

  • How to setup and develop the extension locally
  • How to deploy the extension to production so other Looker users in your Looker instance can use it
  • How to optionally finetune the extension's performance and extend its functionality.
  • How to manage your deployed extension in production

What you'll need

  • A Looker instance, either through a Looker Original License, active Looker Core Trial or an active Looker Core license.
  • develop and deploy permissions on your Looker instance.
  • Permissions to edit a dashboard you would like to try out with the extension.
  • A Looker API key from your Looker instance.
  • A Google Cloud project with billing enabled on it.
  • Cloud Run API, Vertex AI API, and Artifact Registry API enabled on the project.
  • Access to a local environment with gcloud CLI installed. The codelab steps assume a Linux style environment.

2. Setup backend for local development

In this section, you will set up the websocket backend service for you to try out and locally develop. The service will have access to Vertex AI.

  1. Install Node version 18 or higher installed in your local environment. Follow these directions to install Node.
  2. Clone the extension's repository to your local home directory and navigate into the repository's root directory. For the purpose of this codelab all code samples will assume your cloned repository is in your local home directory.
cd ~
git clone git@github.com:looker-open-source/dashboard-summarization.git
  1. Navigate into the cloned repository's root directory and rename the .env.example file to .env to allow you to set environment variables in the later sections of this codelab.
cd ~/dashboard-summarization
mv
.env.example .env
  1. Navigate into the cloned repository's web socket backend's src directory. This directory contains the source code for the server.
cd ~/dashboard-summarization/websocket-service/src   
  1. Install the service's dependencies with NPM.
npm install  
  1. Rename the file looker-example.ini to looker.ini.
mv looker-example.ini looker.ini  
  1. Inside the looker.ini file update:
  2. The client_id and client_secret with the ones from your Looker API key.
  3. The base_url with your Looker instance's URL in the format of: https://<YOUR_LOOKER_URL_MINUS_PROTOCOL>:19999
  4. The text between the brackets (the section header) with the host of your Looker instance's URL.

For example, if your client ID is ABC123, your client secret is XYZ789, and your Looker instance's URL is https://mycompany.cloud.looker.com, your looker.ini file will look exactly like this:

[mycompany]
base_url=https://mycompany.cloud.looker.com:19999
client_id=ABC123
client_secret=XYZ789
verify_ssl=true
  1. Determine your Google Cloud project ID and set it on your PROJECT environment variable. Replace YOUR_PROJECT_ID with your project id.
export PROJECT="YOUR_PROJECT_ID"
  1. Vertex AI provides the Gemini model in multiple regions listed here. Determine which region that your local backend will send and receive data from Vertex AI's Gemini model. Set the region on your REGION environment variable. Replace YOUR_VERTEX_REGION with your region, such as us-central1.
export REGION="YOUR_VERTEX_REGION"
  1. Now start up your local service.
npm start
  1. Your local websocket backend service will be running on http://localhost:5000.

You have now finished setting up the websocket backend service in your local environment!!

The service serves as an interface between your frontend extension and Vertex AI's Gemini model. The service will take dashboard and LookML data from your frontend extension with data queried from Looker and prompt Vertex AI's Gemini model. Then the service will stream Gemini's response to the frontend extension to be displayed in your dashboard.

You can also make changes to the backend service's source code. You will need to first stop the service process, make code changes, then run npm start again.

3. Setup frontend for local development

You will set up the frontend extension for you to try out and develop locally in this section.

  1. In the same local environment from the previous steps, navigate to your cloned repository's root directory and install the frontend server's dependencies for your frontend.
cd ~/dashboard-summarization
npm install
  1. Start up your local frontend development server
npm run develop
  1. Your local frontend server is now serving the extension's javascript at http://localhost:8080/bundle.js.
  2. Open a web browser, and login into your Looker instance.
  3. Follow these instructions to set up a blank LookML project. Name the project dashboard-summarization. You should now have the blank LookML project automatically open in the Looker IDE in your current browser tab.
  4. Create a project manifest file in the root of the LookML project. The file will be called manifest.lkml. If you don't know how, follow these instructions on adding a file to a LookML project.
  5. Replace the content of the new manifest.lkml file with the contents of manifest.lkml in your closed repository's root directory. Select the "Save Changes" button in the top right corner to save the changes to the file.
  6. In a separate browser tab, navigate to the list of database connections in your Looker instance. Follow these instructions if you don't know how.
  7. Choose the Name of one Looker database connection. It does not matter which connection you choose. If you don't have permission to see the database connections, contact your Looker admin and ask for the name of one Looker database connection.
  8. Navigate back to the browser tab with your LookML project opened in a Looker IDE. Create a model file in your LookML project and name the file dashboard-summarization.
  9. Replace the content of your dashboard-summarization.model.lkml file with the below code sample. Make sure to replace the string within the double quotes with the database connection name you chose in step 9. Save the changes to the file.
connection: "<YOUR_CONNECTION_NAME>"
  1. Setup a repository to save your project to. Select the "Configure Git" button in the top right. Select "Set up a bare repository instead." Select "Create Repository".
  2. You now have a basic bare repository to store you LookML project files in. Navigate back to the project in the Looker IDE by selecting "Back to project" or manually navigating back.
  3. Select the "Validate LookML" button in the top right corner. The button will change to "Commit changes and push".
  4. Select the ""Commit changes and push" button. Add any message you want and select "Commit".
  5. Select "Deploy to Production" in the top right corner of the Looker IDE. You've now added the extension to your Looker instance!
  6. Navigate to a Looker dashboard you would like to add the extension to.
  7. Follow the instructions to add an extension tile to your dashboard. Add your new extension to your dashboard as a tile.
  8. Ensure your local websocket backend service you set up earlier is running.

Congratulations! You can now try out the Looker Dashboard Summarization Extension in your dashboard. Your extension will send your dashboard's metadata to your local websocket backend service and will display the Gemini output from your backend service right within your dashboard extension tile.

While your local frontend server is running, you can make changes to the extension's javascript source code and the server will automatically build and serve the changes. You will need to reload your extension or dashboard page to see the changes.

4. Deploy backend to production

In this section, you will now set up the websocket backend service to serve any instances of your dashboard summarization extension on any dashboard in your Looker instance. This will allow other Looker users to try out the extension in their own dashboards without needing to set up their own backend service. These steps assume you have already successfully deployed the backend for local development in the same local environment before.

  1. Follow these instructions to set up an Application Default Credential in your local environment with your project id for the next steps.
  2. Create an Artifact Registry repository for your backend service's Docker images. Replace YOUR_REGION with the region you want your repository to be in.
gcloud artifacts repositories create dashboard-summarization-repo \
   
--repository-format=docker \
   
--location=YOUR_REGION \
  1. Navigate into your cloned repository's web socket backend's src directory.
cd ~/dashboard-summarization/websocket-service/src
  1. Edit the cloudbuild.yaml file and replace all instances of YOUR_REGION and YOUR_PROJECT_ID with your region and project id. Save the changes to the file.
  2. Submit a build using Cloud Build that will build the backend service Docker image and push it up into the Artifact Registry repo you just created. Replace YOUR_REGION with the region you want to use the Cloud Build service in.
gcloud builds submit --region=YOUR_REGION --config cloudbuild.yaml
  1. Remember, your newly built Docker image url is at YOUR_REGION-docker.pkg.dev/YOUR_PROJECT_ID/dashboard-summarization-repo/websocketserviceimage:latest. Replace YOUR_PROJECT_ID with your project id. Replace YOUR_REGION with the region from step 2 you used to create your Artifact Registry repository.
  2. Navigate to the websocket-service/terraform directory in your cloned repository.
cd ~/dashboard-summarization/websocket-service/terraform
  1. Determine which location of Google Cloud Run you want to run your websocket backend service. Choose from these locations.
  2. Edit the variables.tf file and replace YOUR_PROJECT_ID and YOUR_DOCKER_IMAGE_URL with the appropriate values. Check step 6 for your docker image url. Replace YOUR_REGION with the region you chose in the previous step 8. Save your changes to the file.
  3. Deploy the resources your backend service will use using terraform.
terraform init
terraform plan
terraform apply
  1. Save the Deployed Cloud Run URL Endpoint for the next section.

Congrats! You've deployed your websocket backend service and it's now running on Google Cloud Run. Now any instances of your Looker Dashboard Summarization extension can communicate with your backend service. We recommend you always have at least one instance of your websocket backend service up running on Cloud Run. The persistence of your backend service maintains the integrity of the data streaming between your websocket backend service and your extension frontend and helps maintain each user's session as they use your extension.

5. Deploy frontend to production

For this last section, you will do the final steps of deploying the extension frontend to be available to all Looker users in your Looker instance.

  1. Navigate to your cloned repository's root directory.
cd ~/dashboard-summarization
  1. Edit the .env file. Replace YOUR_CLOUD_RUN_URL with the Cloud Run URL Endpoint from the previous section. Save your file changes. This will point the production extension frontend to your websocket backend service running on Cloud Run.
  2. Build the extension's javascript. A dist directory will be automatically generated with a bundle.js file and other files in it.
npm run build
  1. Open a web browser, and login into your Looker instance. Open the side left nav and turn on the "Development Mode" toggle at the bottom.
  2. With the side left nav open, select "Develop", then scroll down and select "dashboard-summarization", your extension's LookML project. You should now be in the Looker IDE for the LookML project.
  3. Drag and drop all the files in the previously generated dist directory into the project's root directory in the "File Browser". Follow these instructions if you need more help.
  4. Open the manifest.lkml file inside the Looker IDE. Inside the file, replace the line
url: "http://localhost:8080/bundle.js"

with

file: "bundle.js"

Replace YOUR_CLOUD_RUN_URL with the Cloud Run URL Endpoint from the end of the last section. Save changes to the file.

  1. Select the "Validate LookML" button in the top right corner. The button will change to "Commit changes and push".
  2. Select the ""Commit changes and push" button. Add any message you want and select "Commit".
  3. Select "Deploy to Production" in the top right corner of the Looker IDE.

Congrats! You've now enabled all Looker users in your Looker instance to add the Looker Dashboard Summarization Extension to their dashboards. As other Looker users use the extension, all instances of the extension will make calls to your deployed websocket backend service running on Google Cloud Run.

Keep in mind, if you make any changes to the source code you must:

  1. Build your extension's javascript again
  2. Replace the generated files you had added to the LookML project with the newly generated files from the dist directory.
  3. Validate, commit, and deploy the LookML project changes to production

Try out the Looker Dashboard Summarization Extension! We encourage you to contribute to the extension and help it better serve the needs of the Looker community. Please feel free to create a pull request on the repository.

Check out the following optional sections to enable Slack/Google Chat exporting, fine tune Gemini's summaries and next steps, and set up Gemini logging.

6. [Optional] Setup export capabilities

Now that you and your Looker users have tried out the Looker Dashboard Summarization extension let share the extension's insight with a greater audience. Follow this section to enable your extension to send summaries and next steps to Google Chat or Slack. You should be familiar with Oauth setup to continue with this section of the codelab.

Enable Google Chat export

  1. Enable the Chat API in your Google Cloud project.
  2. Follow step 1 of the Google Workspace OAuth setup instructions. For scopes you must include spaces.messages.create.
  3. Follow step 2 of the Google Workspace OAuth setup instructions. Add your looker instance's url as a URI under "Authorized JavaScript origins", for example https://mycompany.cloud.looker.com. Note the generated Client ID.
  4. Determine the ID of the Google Chat space you want to export the summaries to. If you're not sure how, follow these directions.
  5. Edit the .env file. Replace YOUR_GOOGLE_CLIENT_ID with the Client ID. Replace YOUR_GOOGLE_SPACE_ID with the Google Chat space ID. Save your file changes. This will configure your extension's frontend to be able to send its insights to the Google Chat space you want.
  6. If you are running your extension's frontend locally, re-build your extension. Else if you are deploying your extension's frontend, re-deploy your extension's frontend.

Enable Slack export

  1. Follow steps 1 and 2 of the official Slack developer docs to setup an OAuth Application. For scopes you must include chat:write and channels:read. Note the generated client ID and client secret.
  2. Determine the ID of the Slack channel you want to export the summaries to.
  3. Edit the .env file. Replace YOUR_SLACK_CLIENT_ID with the Client ID. Replace YOUR_SLACK_CLIENT_SECRET with the Client Secret. Replace YOUR_SLACK_CHANNEL_ID with the channel id. Save your file changes. This will configure your extension's frontend to be able to send its insights to the Slack channel you want.
  4. If you are running your extension's frontend locally, re-build your extension. Else if you are deploying your extension's frontend, re-deploy your extension's frontend.

Now your extension can export its summaries directly to Slack or Google Chat. Keep in mind, the extension can only send summaries to a specific hardcoded Google chat space or Slack channel. You can add additional Oauth scopes and modify the code to fetch and display a list of spaces and channels to send summaries to.

7. [Optional] Finetune the summary and next steps

The extension prompts the Gemini model with all of the dashboard's metadata and queries data. You can improve the accuracy, detail and depth of the summaries and prescriptive steps by adding as much metadata and context to the dashboard itself. Try these steps out for each dashboard your extension is a part of:

  • Follow these directions to add dashboard details to the dashboard. This will help inform the LLM of the general context of the dashboard.
  • Follow these directions to add notes to each dashboard's tile. This will help inform the LLM of the context of each individual query on the dashboard. The small contextual notes will factor into the summaries generated.

The more information you can add to your dashboards the better the extension's summaries and next steps. You can modify the code to include additional dashboard metadata in the prompt to the Gemini model.

8. [Optional] Setup Gemini model logging

Every time a user tells the extension to create summaries for a dashboard, the extension will make a call to Vertex AI for each query in the dashboard plus one final call to format all the summaries. Follow this section to log the Vertex AI calls your extension makes so you can estimate and monitor Vertex AI costs and traffic. You should only follow these directions if you have deployed the websocket backend service.

  1. Determine the Cloud Run location of your deployed websocket backend service.
  2. Follow these instructions to set up a log sink that will route logs to Big Query. The sink destination should be BigQuery. Set the inclusion filter with the following code sample with YOUR_CLOUD_RUN_LOCATION replaced with the previous step's Cloud Run location.
resource.type = "cloud_run_revision"
resource.labels.service_name = "websocket-service"
resource.labels.location = "YOUR_CLOUD_RUN_LOCATION"
 severity>=DEFAULT
jsonPayload.component="dashboard-summarization-logs"

9. Congratulations!

You have set up the Looker Dashboard Summarization extension locally for you to try out. You have also deployed the extension to Google Cloud so other users can try it out too! Now you and other users can access Gemini powered summaries and next steps right from within your dashboards.

10. What's next