Actions on Google is a developer platform that lets you create software to extend the functionality of the Google Assistant, Google's virtual personal assistant, across more than 500 million devices, including smart speakers, phones, cars, TVs, headphones, and more.

Users engage Google Assistant in conversation to get things done, like buying groceries or booking a ride. (For a complete list of what's possible now, see the Actions directory.) As a developer, you can use Actions on Google to easily create and manage delightful and effective conversational experiences between users and your own 3rd-party fulfillment service.

This codelab is part of a multi-module tutorial. Each module can be taken standalone or in a learning sequence with other modules. In each module, we'll provide you with end-to-end instructions on how to build Actions from given software requirements and how to test your code. We'll also teach the necessary concepts and best practices for implementing Actions that give users high-quality conversational experiences.

This codelab covers beginner-level concepts for developing with Actions on Google. You do not need to have any prior experience with the platform to follow this codelab.

What you'll build

In this codelab, you'll build a simple conversational Action with these features:

What you'll learn

What you'll need

The following tools must be in your environment:

Optional: Get the sample code

You can optionally get the full project code for this codelab from our GitHub repository.

So, how does an Action work?

To start a conversation, the user needs to invoke your Action through the Assistant. Users say or type a phrase like "Hey Google, talk to Google IO 18". This tells the Assistant the name of the Action to talk to.

From this point onwards, the user is talking to your Action and giving it input. This conversation continues as a two-way dialog until the user's intent is fulfilled or the conversation is finished.

Let's see what happens behind the scenes during a conversation:

You create Actions by creating an Actions project and defining two things for each action: the intent and the fulfillment.

This section of the codelab describes how you set up your Actions project in the Actions Console.

Check your Google permission settings

In order to test the Action that you'll build for this codelab, you need to enable the necessary permissions.

  1. Go to the ‘Activity Controls' page (https://myaccount.google.com/activitycontrols).
  2. Sign in with your Google account, if you have not already done so.
  3. Ensure that the following permissions are enabled:

Create an Actions project

Actions projects are containers for your Action(s) with the metadata (name, description, category) that becomes your Actions directory listing.

To start building Actions, you'll first need to create an Actions project as follows:

  1. Open the Actions Console.
  2. Click on Add/import project.
  3. Type in a Project name, like "actions-codelab". This name is for your own internal reference; later on, you can set an external name for your project.

  1. Click Create Project.
  2. Rather than pick a category, click Skip on the upper-right corner.

  1. Click Build > Actions in the left nav.
  2. Click Add your first Action.
  3. Select at least one language for your Action, followed by Update. For this codelab, we recommend only selecting English.
  4. On the Custom intent card, click Build. This will open the Dialogflow Console in another tab.

How Actions work with Dialogflow

You might be wondering how the Assistant parses the semantic meaning of user input (such as spoken utterances). This is done via natural language understanding (NLU), which allows Google's software to recognize words in speech.

For your own Actions, Google provides a service called Dialogflow to let you handle NLU easily. Dialogflow simplifies the task of understanding user input, extracting key words and phrases from the input, and returning responses. You define how all this works within a Dialogflow agent.

The following diagram shows an example of how Dialogflow (represented by the 2nd column from the right) handles user input from the Assistant and sends requests to your fulfillment (represented by the rightmost column).

Create a Dialogflow agent

Now that you've built your Actions project, create a Dialogflow agent and associate it with your project:

  1. After following the steps above, you should already be in the Dialogflow Console with your Actions project name at the top. You may need to authorize Dialogflow to use your Google account, and accept the Terms of Service.
  1. Click Create.

If the agent creation is successful, you will be in the Intents page. You can now begin customizing how your Dialogflow agent responds to user requests.

Users start the conversation with your Actions through invocation. (An example of an explicit invocation is a phrase like "Hey Google, talk to MovieTime".) The Assistant then tries to match the user's input to one of your Actions.

Create a welcome intent

Every Actions project must have a welcome intent that acts as an entry point for users to start conversations. The welcome intent is triggered when users explicitly invoke an Action by uttering its name.

By default, Dialogflow creates a welcome intent for us. For this codelab, you'll modify the welcome intent that users trigger when they say "Hey Google, talk to my test app".

To modify the welcome intent:

  1. In the Intents page of the Dialogflow Console, click on Default Welcome Intent.

  1. Delete all of the other text responses by clicking the trash icon next to each one.

  1. Under the Responses section, click Enter a text response and type "Welcome! What is your favorite color?"
  2. Click Save. You should see a "Intent saved" notification briefly pop up.

Test the welcome intent

The Actions Console provides a web interface for testing called the simulator. This interface lets you simulate hardware devices and their settings. You can also access debug information such as the request and response that your fulfillment receives and sends.

To test out your Action in the Actions console simulator:

  1. In the Dialogflow Console left navigation, click on Integrations. Then, click on Google Assistant > Integration Settings.
  2. Click Test to update your Actions project and load it into the Actions Console simulator. (If you see a ‘Check auto-preview setting' dialog, you can leave the ‘Auto-preview changes' option is enabled, then click Continue.)

  1. To test your Action in the simulator, type "Talk to my test app" into the Input field and hit enter.

Notice that when your Action's welcome intent is invoked, the Assistant responds with the greeting message and prompts the user to provide their favorite color.

Debug your Action

The panel on the right shows debugging information that's useful when testing your Action:

In the next section of this codelab, we'll show how you can create dynamic, conversational responses.

Now that users have a way to start a conversation, let's keep the ball rolling.

In this section, you'll learn the basics of setting up slot-filling, an important capability in Dialogflow that allows you to collect a set of values from the user through natural, back-and-forth conversation. This capability is useful if your Action needs a specific set of parameter values to fulfill the user's intent.

To see slot-filling in action (no pun intended!), you'll create a new intent to see the color response: When the user speaks or types their response, Dialogflow will parse the user's input to extract the information your fulfillment needs—namely, the color—and send this to your fulfillment. Your fulfillment then auto-generates a lucky number to send back to the user.

To implement the response, you'll need to add two components in the Dialogflow Console: a Dialogflow intent and a Dialogflow webhook.

You define Dialogflow intents in your agent to control how you want Dialogflow to process user requests.

In the welcome intent, you asked the user for their favorite color. In this codelab, you'll make an intent to capture this information. To do this, we'll use two specific sections of an intent—namely, training phrases and parameters—to parse the information we need from users.

Create a Dialogflow intent

To create the Dialogflow intent, do the following:

  1. In the Dialogflow Console, close the Google Assistant integration settings page if you have it opened. In the left navigation, click on Intents then click on Create Intent.
  2. In the intent name field at the top, type in "favorite color" as the name of your new intent. Click Save.
  1. Under Training phrases, add the following user expressions by typing in the phrases below then clicking Enter on the keyboard:
  1. Under Action and parameters, create the prompt text to ask the user for their favorite color. You should see a line with "color" in the Parameter Name field. This parameter entry is automatically created by Dialogflow because it is able to auto-annotate words and phrases it recognizes as entities in the training phrases. On that line:

  1. Under the Fulfillment section in this page (not the left navigation), click Enable Fulfillment. You may need to expand the section by clicking on the down pointing arrow.
  2. Turn on Enable webhook call for this intent.
  3. Click Save.

After you've completed these steps, Dialogflow will automatically extract parameters it recognizes from user speech which triggers this intent; in this case, the color parameter. Once it obtains this information, Dialogflow makes it available to your fulfillment.

In the next section of this codelab, we'll discuss how to create a Dialogflow webhook that processes the parameter value and handles the intent.

In this section, we'll walk you through the steps of implementing a Dialogflow webhook that contains the logic to construct a conversational response. You'll also be able to try out your Action in real-time by using the Actions Console simulator.

The Dialogflow webhook contains the logic for dynamically constructing response messages to send to the user.

Build your webhook

We'll use the Dialogflow Console's Inline Editor to build and deploy the webhook. To build a webhook for your Action, do the following:

  1. In the left navigation, click on Fulfillment.
  2. Toggle Inline Editor to enabled.

  1. Replace the contents of index.js with the following code:

index.js

'use strict';

// Import the Dialogflow module from the Actions on Google client library.
const {dialogflow} = require('actions-on-google');

// Import the firebase-functions package for deployment.
const functions = require('firebase-functions');

// Instantiate the Dialogflow client.
const app = dialogflow({debug: true});

// Handle the Dialogflow intent named 'favorite color'.
// The intent collects a parameter named 'color'.
app.intent('favorite color', (conv, {color}) => {
    const luckyNumber = color.length;
    // Respond with the user's lucky number and end the conversation.
    conv.close('Your lucky number is ' + luckyNumber);
});

// Set the DialogflowApp object to handle the HTTPS POST request.
exports.dialogflowFirebaseFulfillment = functions.https.onRequest(app);
  1. Click Deploy. You'll need to wait on this page a couple minutes for Dialogflow to provision and deploy your webhook code. When the code is successfully deployed, you'll see an updated "Last deployed" timestamp next to the Deploy button.

Understand the code

The code for the webhook is implemented in JavaScript. When you use Dialogflow's inline editor, it deploys your webhook code on a managed environment in the cloud using Cloud Functions for Firebase.

The webhook code uses the Actions on Google Node.js client library to respond to HTTP requests that the Assistant sends to your webhook. The library allows you to create a DialogflowApp object, which acts as a wrapper for the Dialogflow API.

In this case, an app variable of the DialogflowApp object type is created. The app.intent() function is used to declare a callback to handle the ‘favorite color' intent.

The callback receives two important arguments:

The callback generates the user's lucky number by determining the length of the word for their color (for example, the 6-character word "orange" would return a favorite color of 6).

It then uses the close method of the Conversation object parameter to respond to the user and end the conversation. The response is a simple string (e.g. "Your lucky number is 6").

The figure above shows how the variable color on line 21 of the webhook implementation uses the parameter name we defined earlier in the Dialogflow console. For example, if the user said "My favorite color is green", then our variable color on line 21 will have the value "green".

Test your Action

To test your Action:

  1. Open the Actions Console and select your Actions project for this codelab.
  2. Click on Simulator in the left navigation. Click on the X button at the top of the output window. This exits you from any conversation you might have started earlier in the simulator.
  3. Type "Talk to my test app" into the Input field and hit enter.

The simulator will then perform an explicit invocation by calling your welcome intent. Once you're mid-conversation, you can trigger the ‘favorite color' custom intent by providing a color. For example, try testing with the following phrases:

Congratulations!

You now know the basics of building conversational user interfaces with Actions on Google.

What we've covered

What's next

In the next codelab, you'll make further refinements to your Actions project. You'll also learn more about developing webhooks, adding additional actions, and implementing contextual state management.

You can also explore these resources for learning about Actions on Google:

Follow us on Twitter @ActionsOnGoogle to stay tuned to our latest announcements, and tweet to #AoGDevs to share what you have built!

Feedback survey

Before you go, please fill out this form to let us know how we're doing!