Actions on Google is a developer platform that lets you create software to extend the functionality of Google Assistant, Google's virtual personal assistant, across more than 1 billion devices, including smart speakers, phones, cars, TVs, headphones, and more. Users engage Google Assistant in conversation to get things done, like buying groceries or booking a ride (for a complete list of what's possible now, see the Actions directory.) As a developer, you can use Actions on Google to easily create and manage delightful and useful conversational experiences between users and your own 3rd-party service.

This codelab is an advanced module and is intended for readers who already have some experience building Actions for Google Assistant. If you don't have any prior development experience with Actions on Google, we strongly recommend that you familiarize yourself with the platform by following our introductory codelabs (Level 1, Level 2, and Level 3). These modules will guide you through a series of features that can help you expand your Action's functionality and grow your audience.

In this codelab, you use Interactive Canvas, a framework built on Google Assistant, to add a full-screen game to a Conversational Action. The game is an interactive web app that Assistant sends as a response to the user in conversation. The user can then play the game through either voice or text input on smart displays and Android mobile devices.

Instead of building an entire game yourself, you deploy a partially prebuilt game called Snow Pal and add the game's logic as you progress through the codelab. Snow Pal is a variation of the traditional Hangman game; the game presents you with a number of blank spaces representing a word, and you guess letters that you think may be in the word. With each incorrect guess, your Snow Pal melts a bit more. You win the game if you can figure out the word before your Snow Pal melts completely.

Figure 1. A partially completed Snow Pal game.

What you'll build

In this codelab, you build an Action that uses Interactive Canvas. Your Action will have the following features:

When you've finished this codelab, your completed Action will have the following conversational flow:

Assistant: Welcome to Snow Pal! Would you like to start playing the game?

User: Yes

Assistant: Try to figure out the word by guessing letters that you think are in the word. Figure out the word before the Snow Pal melts to win the game! After 5 incorrect guesses, the Snow Pal melts and the game is over. If you know the word, you can say, for instance "The word is penguin." You can try another word, or ask for help.

User: I guess the letter E

Assistant: E is right. Easy peasy lemon squeezy!

The user continues guessing letters, or guessing the unknown word itself, until the game ends.

What you'll learn

What you'll need

Optional: Get the full sample code

In this codelab, you build the sample step-by-step from an incomplete version. If you prefer, you can also get the completed sample from the GitHub repository.

To understand how Interactive Canvas Actions work, familiarize yourself with the following terms:

An Action that uses Interactive Canvas works similarly to a regular Conversational Action. The user still has a back-and-forth conversation with the Action to fulfill their goal; however, instead of returning responses in-line in the conversation, an Interactive Canvas Action sends a response to the user that opens the full-screen web app. The Interactive Canvas API allows the Conversational Action and web app to communicate with each other (in this way, the user's input can update the web app game accordingly). The user continues to interact with the web app through voice or touch until the conversation is over.

The following diagram shows how Interactive Canvas Actions work at a high level after invocation:

  1. The Action prompts the user to guess a letter in the word or the word itself.
  2. The user says "I guess the letter i" to the smart display.
  3. The user's input is routed to your Conversational Action, which contains your Action's conversational logic.
  4. The logic in the Conversational Action processes the user's input and triggers other logic to update the web app based on what the user said. In this example, the game on the smart display updates to show the letter "i" in the word.
  5. The conversational Action responds to the user that their guess was correct. The user can now guess another letter or the word itself.

The following sections describe how to set up your development environment and create your Actions on Google project.

Install the Firebase CLI

The Firebase Command Line Interface (CLI) allows you to deploy your Actions project to Cloud Functions and host your web app.

Note: If you have already installed the Firebase Command Line Interface, you can skip these steps and proceed to the next section.

To install or upgrade the CLI, open a terminal and run the following npm command:

npm install -g firebase-tools

To verify that the CLI has been installed correctly, open a terminal and run the following command:

firebase --version

Make sure the version of the Firebase CLI is 3.5.0 or higher, so that it has all the latest features required for Cloud Functions. If it's not, run npm install -g firebase-tools to upgrade.

To log into Firebase, run the following command:

firebase login

When you log in to Firebase, make sure that you use the same Google account that you use to create your Actions project.

Clone the repository

In this section, you clone the files you need for this codelab. To get these files, follow these steps:

  1. Open a terminal and change to a directory where you usually store coding projects. If you don't have one, navigate to your home directory.
  2. To clone this repository, run the following command in your terminal:
git clone https://github.com/actions-on-google/dialogflow-interactive-canvas-codelab-nodejs

Open the start/ directory. This repository contains the following important files that you edit throughout this codelab:

You learn more about the purpose of these files later in this codelab.

Check your Google permission settings

In order to test the Action that you build for this codelab, you need to enable the necessary permissions. To enable these permissions, follow these steps:

  1. Go to the Activity Controls page.
  2. Sign in with your Google account, if you have not already done so.
  3. Enable the following permissions:

Create Actions project

Your Actions project is a container for your Action. To create your Actions project for this codelab, follow these steps:

  1. Open the Actions console. If you've not already done so, sign in with the same Google account you used to log into Firebase.
  2. Click New project.
  3. Name your project Snow Pal.
  4. Click Create project.
  5. Click the Games & fun card.
  6. Click the Conversational card.
  7. Click Deploy in the top navigation.
  8. Check the Yes option under Do your Actions use Interactive Canvas?
  9. At the top of the page, click Save.

Import Dialogflow agent

One natural language understanding solution for your Action is to create and use a Dialogflow agent, which is a project that you customize to converse with your Action users. Your Dialogflow agent maps things that the user says to functions in your fulfillment code. (Your fulfillment contains the conversational logic for your Action.)

For this codelab, you import a Dialogflow agent with pre-configured intents. You add some of these intents programmatically to fulfillment later.

To set up your Dialogflow agent and associate it with your Actions project, follow these steps:

  1. Click Overview in the top navigation.
  2. Click the drop-down arrow next to Build your Action. Then, click Add Action(s).
  3. Click Add your first action.
  4. Under Built-in intents, select Play game and click Get started in Dialogflow.

  1. Click Sign in with Google.
  2. On the page titled Create an agent, enter a name for your agent (we suggest using Snow Pal).
  3. Click Create Agent.
  4. A page loads with the text Your Dialogflow agent has been created. On this page, click Go to Agent.
  5. Click the gear icon in the left navigation next to the name of your agent.

Note: Ignore the warning about not saving.

  1. Click Export and Import.
  2. Click Restore from Zip.
  3. Upload the agent.zip file from the start directory.
  4. Type RESTORE in the input field and click Restore.
  5. Click Done.

So far, you've done the following:

In the following sections, you deploy both your conversational fulfillment and the Snow Pal web app, and connect these elements to the Assistant to create an Action.

Deploy your web app

In this section, you deploy your web app (the Snow Pal game) using the Firebase CLI. Once you deploy, you can retrieve the URL for the web app and see how the game looks in a browser.

Note: You can deploy both your fulfillment files and web app files with the Firebase command firebase deploy. In this codelab, you deploy your web app and fulfillment separately.

To deploy your web app using Firebase, follow these steps:

  1. If you skipped the Install the Firebase CLI section, run firebase login in the terminal to log in to the same Google account that you used to create your Actions project.
  2. In a terminal, run the following command from the root directory:
$ cd start/
  1. Using your Action's project ID, run the following command:
firebase use <PROJECT_ID>

  1. Run the following command in the terminal to deploy the Snow Pal web app to Firebase:
firebase deploy --only hosting

After a few minutes, you should see "Deploy complete!", which indicates that you've successfully deployed the Snow Pal web app to Firebase.

To view the Snow Pal game in a browser, follow these steps:

  1. Retrieve the Hosting URL provided in the output of your terminal. The URL should be in the following form: https://PROJECT_ID.firebaseapp.com

Note: If the URL doesn't appear in your terminal output, you can retrieve the web app URL with the following steps:

  1. Open the Firebase Console.
  2. Select your Actions project from the list of options.
  3. Navigate to Develop > Hosting on the left navigation bar.
  4. Under the Dashboard tab, you should see a list of domains for your project. Copy the URL. It should be in the following form: https://PROJECT_ID.firebaseapp.com
  1. Paste the URL in a browser. You should see the ‘Snow Pal' game starting screen with a Start Game button (this button doesn't work yet):

Use intents with Interactive Canvas

In regular Conversational Actions, a user's input maps to a specific intent, which then delivers the appropriate response to the user. With Interactive Canvas, intents still recognize a user's input and respond appropriately; however, intents must also contain information to update the web app. For example, if a user guesses the letter "i" in the Snow Pal game, an intent could only recognize the input and respond, Good job, that's correct! However, the user can only play the game if the web app also updates to show the letter "i" in the appropriate blank space of the word. To update the web app based on a user's input, intents in Interactive Canvas must contain an HtmlResponse.

Add Welcome intent

In this section, you add the logic for the Welcome intent to the /functions/index.js file, which contains your conversational fulfillment. Similar to the other intents in this codelab, the Welcome intent already exists in the Dialogflow agent you imported, but you must add it to fulfillment for it to work. The Welcome intent is matched when a user invokes an Action and typically introduces the user to the Action.

For Interactive Canvas, the Welcome intent contains the URL for the web app that renders the game. When the user's input matches the Welcome intent, the specified URL opens.

To add the Welcome intent logic to your fulfillment, copy and paste the following code snippet into the /functions/index.js file under the /start/functions/ directory:

index.js

const WELCOME_BACK_GREETINGS = [
 `Hey, you're back to Snow Pal!`,
 `Welcome back to Snow Pal!`,
 `I'm glad you're back to play!`,
 `Hey there, you made it! Let's play Snow Pal.`
];

app.intent('Welcome', (conv) => {
 if (conv.user.last.seen) {
   conv.ask(`${randomArrayItem(WELCOME_BACK_GREETINGS)} Would you like to start playing the game?`);
 } else {
   conv.ask(`Welcome to Snow Pal! Would you like to start playing the game?`);
 }
 // Replace PROJECT_ID with the ID for your project.
 conv.ask(new HtmlResponse({
   url: `https://PROJECT_ID.firebaseapp.com`,
 }));
});

Here, you set up the Welcome intent so that, if the user has played the game before, they receive a shorter welcome message than a new user. Within the Welcome intent, you create a new instance of the HtmlResponse class that contains the URL for the web app that renders the game. In the url field, paste the URL of your deployed web app that you retrieved in the Deploy your web app section. When the user invokes the Snow Pal Action, this URL opens the web app on the device with a display.

Deploy your fulfillment

You now deploy your newly modified /functions/index.js file and other files in the /start/functions directory to Cloud Functions with the Firebase CLI. The deployed /functions/index.js file serves as your fulfillment, which contains the conversational logic for your Action.

To deploy your fulfillment, follow these steps:

  1. In a terminal, navigate to the start/functions directory of your project.
  2. Run the following command in the terminal to install dependencies:
npm install
  1. Run the following command in the terminal to deploy your webhook code to Firebase:
firebase deploy

After a few minutes, you should see "Deploy complete!", which indicates that you've successfully deployed your webhook to the cloud.

Add your fulfillment URL to Dialogflow

Next, you need to provide Dialogflow with the URL to the Cloud Function.

To retrieve this URL, follow this step:

If the URL does not appear in your terminal output, follow these steps to retrieve the URL:

  1. Open the Firebase Console.
  2. Select your Actions project from the list of options.
  3. Navigate to Develop > Functions on the left navigation bar. If you're prompted to "Choose data sharing settings", click Do this later.
  4. Under the Dashboard tab, you should see an entry for dialogflowFirebaseFulfillment with a URL under Trigger. Copy this URL.

To update your Dialogflow agent to use your webhook for fulfillment, follow these steps:

  1. Open the Dialogflow console (you can close the Firebase console if you'd like).
  2. Click Fulfillment in the left navigation.
  3. Toggle the Webhook slider to Enabled.
  4. In the URL field, paste the URL you copied from the Firebase dashboard if it doesn't already appear.
  5. Click Save.

Test in simulator

To test your Action throughout this codelab, you use the Actions on Google simulator. The simulator lets you simulate a conversation between your user and your Action for a variety of hardware devices.

To test out your Action in the Actions console simulator, follow these steps:

  1. In the Dialogflow console left navigation, click Integrations. Then, click Google Assistant > Integration Settings.
  2. Click Test to update your Actions project and load it into the Actions console simulator. If you see a ‘Check auto-preview setting' dialog, you can leave the ‘Auto-preview changes' option enabled, then click Test.

  1. Set the surface to Smart Display.

  1. Type Talk to my test app into the Input field and press enter. This invocation begins a conversation with your Action.

When your Action's Welcome intent is invoked, Assistant responds with the greeting message, "Welcome to Snow Pal! Would you like to start playing the game?" The Snow Pal game appears under the Display tab on the right.

Now that you've successfully deployed your web app and fulfillment, you can build out the logic for the Snow Pal game in the following sections.

In the previous section, you created an Action that renders the Snow Pal web app game when you invoke the Action through voice. In the following sections, you extend your Action so that users can play the Snow Pal game with their voice.

To build the logic for the game, you edit the following files under the /start folder:

Before adding intents to enable users to play the game, you must register and configure callbacks to update the Snow Pal game based on a user's input.

Use callbacks to pass data between a web app and fulfillment

In this part of the codelab, you create intents in your fulfillment file that, when matched, pass data to your web app to update it accordingly. For example, if the user guesses a letter in the Snow Pal game that exists in the word, the web app updates to show the letter in the correct position in the word.

While the following sections go into more detail, the general flow for how data is passed from fulfillment to the web app is the following:

  1. The user matches an intent that includes an HtmlResponse.
  2. The fulfillment sends the HtmlResponse, which triggers the onUpdate() callback.
  3. The onUpdate() callback maps to custom logic that updates the web app accordingly.

HtmlResponse and data

In Part 4, you learned how an HtmlResponse can contain information that renders or updates the web app. You added the Welcome intent to your fulfillment, which includes an HtmlResponse with a url attribute that contains the web app URL and renders it.

HtmlResponse has another attribute, data, that you can use to pass data from your fulfillment to your web app. The following example sends JSON data that contains a string called state with the string value of NEW_GAME, which maps to logic that starts a new game:

conv.ask(new HtmlResponse({
  data: {
    state: 'NEW_GAME'
  },
}));

onUpdate() callback

Interactive Canvas Actions use callbacks to respond to information or requests from your conversational Action. The onUpdate() callback updates the web app based on the intent-specific data your fulfillment sends.

In the example HtmlResponse code snippet above, the data value is passed into the onUpdate() callback. The callback then maps the data state attribute to the NEW_GAME command, which triggers other logic that begins a new game.

The updated diagram below shows how HtmlResponse and onUpdate() fit into this Interactive Canvas Action:

  1. The Action prompts the user to guess a letter in the word or the word itself.
  2. The user says "I guess the letter i" to the smart display.
  3. The user's input is routed to Dialogflow to match an intent.
  4. Your fulfillment executes the logic for the matched intent and sends the HtmlResponse to the device.
  5. The HtmlResponse data is passed to the onUpdate() callback of the web app.
  6. The logic for your web app reads the data value of the HtmlResponse and updates the web app appropriately.
  7. The Conversational Action responds to the user that their guess was correct. The user can now guess another letter or the word itself.

Load the Interactive Canvas API

The Interactive Canvas API enables communication between the web app and your conversational Action. You add the script for this API in /public/index.html, which is the main HTML file for the Snow Pal game. This file defines how your UI looks and loads in several scripts.

To load the Interactive Canvas API, add the following lines between the <head> tags of /public/index.html:

index.html

<!-- Load Assistant Interactive Canvas API -->
 <script type="text/javascript" src="https://www.gstatic.com/assistant/interactivecanvas/api/interactive_canvas.min.js"></script>

Declare and register callbacks

Now, you initialize and register Interactive Canvas callbacks to communicate changes between your fulfillment and web app. In /public/js/assistant.js, there is a pre-configured class called Assistant for declaring and registering callbacks.

To include this class in your web app, add the following line between the <head> tags of /public/index.html:

index.html

<script type="text/javascript" src="js/assistant.js"></script>

In the Assistant class of /public/js/assistant.js, add a new function named setCallbacks():

assistant.js

setCallbacks() {
  const that = this;
  // Declare the Interactive Canvas action callbacks.
  const callbacks = {
      onUpdate(data) {

      },
  };
  // Called by the Interactive Canvas web app once web app has loaded to
  // register callbacks.
  this.canvas.ready(callbacks);
}

The setCallbacks() function declares callbacks, including the onUpdate() callback, and passes the callbacks to the ready() method to register them with the Interactive Canvas API.

To register the onUpdate() callback, add the following lines to /public/js/game.js at the end of the create() method:

game.js

// Set assistant at game level.
this.assistant = new Assistant(this);
// Call setCallbacks to register assistant action callbacks.
this.assistant.setCallbacks();

Here, you create a new Assistant instance and call the setCallbacks() method. The create() method is called once to create the game and registers the callbacks during game creation.

Create command map for game updates

Now that you've registered the onUpdate() callback, you can create a command map that will contain functions that update the Snow Pal game based on a user's input. To do so, you update the Assistant class so that it maps state enums to functions to execute. For example, you'll later add the WIN enum, which maps to a function that displays a message on the web app indicating that you've won. In this section, you set up only the DEFAULT enum.

To set up the command map, update the Assistant constructor() method in /public/js/assistant.js with the lines between /* ADD CODE HERE...*/ and /* END */ in the following snippet:

assistant.js

constructor(scene) {
  this.canvas = window.interactiveCanvas;
  this.gameScene = scene;
/* ADD CODE HERE to update command map: */
  const that = this;
  this.commands = {
    DEFAULT: function() {
      // do nothing
    },
  };
/* END */
}

Here, you're adding a JavaScript object called commands, which contains a DEFAULT enum mapping to an empty function.

Next, you can update the onUpdate() callback to select which function to execute based on the state provided in the data field from fulfillment. To update the callback, add the line between /* ADD CODE HERE...*/ and /* END */ in /public/js/assistant.js:

assistant.js

setCallbacks() {
  const that = this;
  // Declare the Interactive Canvas action callbacks.
  const callbacks = {
    onUpdate(data) {
  /* ADD CODE HERE to update callback: */
      that.commands[data.state ? data.state.toUpperCase() : 'DEFAULT'](data);
  /* END */
    },
  };
  // Called by the Interactive Canvas web app once web app has loaded to
  // register callbacks.
  this.canvas.ready(callbacks);
}

At this point, the starting screen for the Snow Pal game appears in the simulator, but you can't yet begin playing the game. This section shows you how to add a Start Game intent that is matched when the user says "Yes" in response to "...Would you like you to start playing the game?" The matched intent then triggers logic that starts a new session of the Snow Pal game.

In this section, you add a Start Game intent handler to your fulfillment and configure the callback that begins a new game session.

Add the Start Game intent

In Part 3: Setup, you imported a pre-built Dialogflow agent that contains the Start Game intent and its training phrases, as well as all other intents for this codelab. You now need to add the logic for this intent in /functions/index.js, which serves as your fulfillment.

Add the following code snippet to /functions/index.js:

index.js

const START_GAME_RESPONSES = [
 `Try guessing a letter in the word, or guess the entire word if you think you know what it is.`,
 `Try guessing a letter in the word, or guess the entire word if you're feeling confident!`,
 `Try guessing a letter in the word or guessing the word.`,
 `Try guessing a letter in the word or guessing the word.`
];
app.intent('Start Game', (conv) => {
 if (conv.user.last.seen) {
   conv.ask(randomArrayItem(START_GAME_RESPONSES));
 } else {
   conv.ask(`${INSTRUCTIONS}`);
 }
 conv.data.incorrectGuesses = 0;
 // Generate new word to guess
 conv.data.correctWord = DICTIONARY.getWord().toLocaleUpperCase();
 conv.data.wordToDisplay = '_'.repeat(conv.data.correctWord.length);
 conv.ask(new HtmlResponse({
   data: {
     state: 'NEW_GAME',
     wordToDisplay: conv.data.wordToDisplay
   },
 }));
});

Here, you create the intent handler for the Start Game intent. When the user's input matches this intent, it responds with instructions for how to play the game and generates a word for the user to guess.

This intent handler includes an HtmlResponse that contains data to pass to the web app to start a new game. The HtmlResponse contains the state string NEW_GAME, which maps to logic for beginning a new game. It also contains wordToDisplay, which updates the web app with the correct amount of dashes representing the unknown word.

Configure callback for Start Game intent

Next, configure your callback to update the game when it receives the NEW_GAME state string.

The state field provided in the data from fulfillment is passed to the onUpdate() callback. The onUpdate() callback then executes the corresponding function defined in the commands map of our Assistant class. Currently, the commands map only contains a DEFAULT enum, which runs when no other state is matched.

To update the commands map to trigger logic that starts a new game, add the lines between /* ADD CODE HERE...*/ and /* END */ in the following snippet to the commands map in /public/js/assistant.js:

assistant.js

this.commands = {
/* ADD CODE HERE for NEW_GAME enum: */
  NEW_GAME: function(data) {
    that.gameScene.start(data.wordToDisplay);
  },
/* END */
  DEFAULT: function() {
    // do nothing
  },
};

The NEW_GAME enum maps to the start() function, which is pre-built and exists in the /public/js/game.js file. The start() function begins a new game, which includes displaying the Snow Pal and the dashes representing the unknown word in wordToDisplay.

Redeploy fulfillment and web app

To redeploy your fulfillment and web app files, run the following command in your terminal from the start directory:

firebase deploy

Test Start Game intent

Now, test in the simulator to check that the game begins when you match the Start Game intent. To test the Start Game intent, follow these steps:

  1. Click Test in the top navigation of the Actions console.
  2. Make sure the surface is set to Smart Display.
  3. The Input field should be pre-populated with Talk to my test app. Click into this field and press enter. This invocation triggers the Welcome intent: "Hey, you're back to Snow Pal! Would you like to start playing the game?"
  4. Type Yes into the Input field and press enter. Your Action should display the game with dashes representing a word and respond with instructions about how to play the game.

Configure the Start Game button

So far, you've configured your Action so that the Snow Pal game begins when you match the Start Game intent. In this section, you configure the Start Game button in your Snow Pal web app to begin the game when the user taps it. By the end of this section, your users will be able to invoke the game through either touch or voice input!

You use sendTextQuery(), an Interactive Canvas API that allows you to trigger an intent through touch input, to make the button work.

In this case, you use sendTextQuery() to trigger the Start Game intent when a user clicks the button. The Start Game intent then triggers logic that updates the web app and begins a new game session.

To implement sendTextQuery(), add the line between /* ADD CODE HERE...*/ and /* END */ in the following snippet to /public/js/game.js:

game.js

this.startButton
  .setInteractive({ useHandCursor: true })
  .on('pointerover', () => this.startButton.setStyle({fill: '#ff0'}))
  .on('pointerout', () => this.startButton.setStyle({fill: '#000'}))
  .on('pointerdown', () => this.startButton.setStyle({fill: '#0ff'}))
  .on('pointerup', () => {
    this.startButton.setStyle({fill: '#ff0'});
   /* ADD CODE HERE to use sendTextQuery(): */
    window.interactiveCanvas.sendTextQuery('Yes');
   /* END */
  });

Here, you add sendTextQuery() to the pre-configured code for the Start Game button. The Yes argument matches a training phrase in the Start Game intent, and triggers this intent in the same way as when a user gives voice input. A new game session then begins.

Redeploy fulfillment and web app

To redeploy your fulfillment and web app, run the following command in your terminal from the start directory:

firebase deploy

Test Start button

To test the Start button, navigate to the simulator in the Actions console and follow these steps:

  1. Make sure the surface is set to Smart Display.
  2. Type Talk to my test app in the Input field and press enter. This invocation triggers the Welcome intent: "Hey, you're back to Snow Pal! Would you like to start playing the game?"
  3. Click the Start Game button that appears on the display. A new game session should begin.

So far, you've added intents and configured callbacks that allow you to start the Snow Pal game through voice or touch input. You now add the game's logic— the Guess Letter or Word intent, which enables you to guess a letter in the word or the complete word itself. This intent contains the logic for determining if the user's guess is correct, if the user has won the game, if the user's guess is incorrect, or if the user has lost the game, and updates the web app accordingly.

In this section, you add the Guess Letter or Word intent handler to your fulfillment and configure the callback for this intent to update the game based on the user's input.

Add the Guess Letter or Word intent

To add the Guess Letter or Word intent to your fulfillment, add the following code snippet to the /functions/index.js file:

index.js

const PLAY_AGAIN_INSTRUCTIONS = 'Would you like to  play again or quit?';

const RIGHT_RESPONSES = ['Right on! Good guess.', 'Splendid!',
 'Wonderful! Keep going!', 'Easy peasy lemon squeezy!', 'Easy as pie!'];

const WRONG_RESPONSES = [`Whoops, that letter isn't in the word. Try again!`,
 'Try again!', 'You can do this!', 'Incorrect. Keep on trying!'];

const WIN_RESPONSES = ['Congratulations and BRAVO!',
 'You did it! So proud of you!',
 'Well done!', `I'm happy for you!`,
 `This is awesome! You're awesome! Way to go!`];

This snippet defines several different responses to the user for various scenarios (for example, when the user guesses a letter correctly or wins the game) to make your Action sound more natural.

Add the following code snippet directly below the constant declarations:

index.js

app.intent('Guess Letter or Word', (conv, {letterOrWord}) => {
 conv.ask(`<speak>Let's see if ${letterOrWord} is there...<break time="2500ms"/></speak>`);
 letterOrWord = letterOrWord.toLocaleUpperCase();

// Add the logic for the intent here.

});

Here, you add the basic handler for the Guess Letter or Word intent. The intent takes the argument letterOrWord, which corresponds to the letter or word the user guesses. If the user guesses "a", the intent responds to the user with "Let's see if a is there...".

Add the following code snippet below the commented line in the previous snippet:

index.js

 // Check if the letter guessed is part of the correct word.
 let correctGuess = conv.data.correctWord.indexOf(letterOrWord) > -1;
if (correctGuess) {
   // Update the word to be displayed to the user with the newly guessed letter.
   updateWordToDisplay(conv, letterOrWord);
   // Check if the correct guess will result in the user winning the game.
   const userHasWon = conv.data.wordToDisplay === conv.data.correctWord;
   if (userHasWon) {
     conv.ask(`<speak>${letterOrWord} is right. That spells ${conv.data.correctWord}! ${randomArrayItem(WIN_RESPONSES)}`  +
     `${PLAY_AGAIN_INSTRUCTIONS}</speak>`);
     // User has won the game. Update game to WIN state, and pass the
     // updated word to display to the front-end.
     conv.ask(new HtmlResponse({
       data: {
         state: 'WIN',
         wordToDisplay: conv.data.wordToDisplay
       },
     }));
   } else {
     conv.ask(`${letterOrWord} is right. ${randomArrayItem(RIGHT_RESPONSES)}`);
     // User made a correct guess. Update game to CORRECT state, and pass the
     // updated word to display to the front-end.
     conv.ask(new HtmlResponse({
       data: {
         state: 'CORRECT',
         wordToDisplay: conv.data.wordToDisplay
       },
     }));
   }
 }

// Add remaining intent logic here

The above snippet contains the logic for determining if the user's guess is correct and if the user has won the game. If the guess is correct, the function updateWordToDisplay() executes, which updates the word with the guessed letter on the backend. The code then compares the displayed word to the correct word to check whether or not the user has won the game. If the user wins, the Action tells the user they have won and asks if they'd like to play again.

The first HtmlResponse instance updates the web app if the user wins. Your fulfillment sends the HtmlResponse to the front-end when the user wins the game, which updates the web app to show a You Win! screen. The HtmlResponse passes the state string WIN to onUpdate(), which maps to the WIN enum in your command map and triggers logic to update the web app (you configure this enum in the following section). This HtmlResponse also contains wordToDisplay, the updated string to display on the web app that represents the letters the user has guessed correctly.

The second HtmlResponse instance updates the web app when the user guesses a letter correctly but has not won the game. In this case, your fulfillment sends the data contained in the HtmlResponse to your front-end, and the web app updates to show the guessed letter in the word. The state string, CORRECT, is passed to onUpdate(), which maps it to the CORRECT enum in your command map. The web app then uses the wordToDisplay string to update the word displayed on the screen.

Next, add the following code snippet at the bottom of the Guess Letter or Word intent:

index.js

else {
   conv.data.incorrectGuesses++;
   // Check if the user has exceeded the maximum amount of max guesses allowed.
   const userHasLost = conv.data.incorrectGuesses >= MAX_INCORRECT_GUESSES
   if (userHasLost) {
     conv.ask(`<speak>Sorry, you lost. The word is ${conv.data.correctWord}.` +
       `${PLAY_AGAIN_INSTRUCTIONS}</speak>`);
     // User has lost the game. Update game to LOSE state.
     conv.ask(new HtmlResponse({
       data: {
         state: 'LOSE',
       },
     }));
   } else {
     conv.ask(`${letterOrWord} is wrong. ${randomArrayItem(WRONG_RESPONSES)}`);
     // User made an incorrect guess. Update game to INCORRECT state.
     conv.ask(new HtmlResponse({
       data: {
         state: 'INCORRECT',
       },
     }));
   }
 }

Here, you add logic to the Guess Letter or Word intent that determines if the user's guess is incorrect and if they've lost the game. If the user exceeds the maximum number of guesses in Snow Pal and loses, the Action tells the user they've lost the game and what the mystery word is. In this case, the HtmlResponse passes the state string LOSE to onUpdate(), which maps to the LOSE enum in your command map.

If the user guesses incorrectly but has not lost the game, your fulfillment passes the instance of HtmlResponse with the state string INCORRECT to the front-end. This string maps to the INCORRECT enum in your command map, and triggers logic that updates the web app to show the disappearing Snow Pal.

Update command map for Guess Letter or Word intent

In the previous section, you configured multiple instances of HtmlResponse to pass state strings to your web app for different scenarios (CORRECT, INCORRECT, WIN, and LOSE). Now, you can modify your commands map to include these enums, and trigger other logic that updates your web app accordingly.

Add the lines between // ADD CODE HERE... and // END in the following snippet to your commands map in /public/js/assistant.js:

assistant.js

this.commands = {
  // ADD CODE HERE to add enums to command map:
  CORRECT: function(data) {
    that.gameScene.correctGuess(data.wordToDisplay);
  },
  INCORRECT: function(data) {
    that.gameScene.incorrectGuess();
  },
  WIN: function(data) {
    that.gameScene.win(data.wordToDisplay);
  },
  LOSE: function(data) {
    that.gameScene.lose();
  },
  // END
  NEW_GAME: function(data) {
    that.gameScene.start(data.wordToDisplay);
  },
  DEFAULT: function() {
    // do nothing
  },
};

Here, each enum maps to a function that updates the web app. For example, CORRECT maps to the function correctGuess() in /public/js/game.js, which plays a happy sound and updates the web app with the correctly guessed letter when it executes. For simplicity, all of the functions the enums map to are pre-built for you in the /public/js/game.js file.

Redeploy fulfillment and web app

To redeploy your fulfillment and web app, run the following command in your terminal:

firebase deploy

Test Guess Letter or Word intent in simulator

To test the Guess Letter or Word intent, navigate to the simulator in the Actions console and follow these steps:

  1. Make sure the surface is set to Smart Display.
  2. Type Talk to my test app in the Input field and press enter.
  3. Type yes and press enter. The Action responds with instructions for how to play the game.
  4. Type a letter to guess and press enter. The Action lets you know whether your guess is correct and updates the web app accordingly. (If your guess is wrong, one of the Snow Pal's arms disappears; if your guess is right, the letter appears in the mystery word.)

Now that you've fully built your Action, you can use an additional Interactive Canvas method to enhance your web app. Interactive Canvas Actions have a reserved space for a header that displays the name of your Action at the top of Smart Displays and Android mobile devices. You can preview the header in the simulator atop the Snow Pal game with the text my test app:

You may notice that the header positioning truncates the snowflakes in the web app. To retrieve the dimensions of the header so that no part of the web app is cut off, you can use an Interactive Canvas method called getHeaderHeightPx().

To obtain the header dimensions and update the display size of the background accordingly, add the following snippet to the create() method of the /public/js/game.js file after the code block beginning with the line this.background:

game.js

const that = this;
window.interactiveCanvas.getHeaderHeightPx()
  .then((headerHeight) => {
      that.background
          .setY(headerHeight)
          .setDisplaySize(that.scale.width, that.scale.height - headerHeight);
  });

This code snippet retrieves the height of the header and reconfigures the web app so that it displays fully under the header.

Redeploy fulfillment and web app

To redeploy your fulfillment and web app, run the following command in your terminal:

firebase deploy

Test getHeaderHeightPx() in simulator

To test the results of applying getHeaderHeightPx(), navigate to the simulator in the Actions console and follow this step:

  1. Make sure the surface is set to Smart Display.
  2. Type Talk to my test app in the Input field and press enter.
  3. Type yes and press enter.

The previously cut-off snowflakes should now be fully visible beneath the header:

In this section, you can apply what you've learned in the codelab to further enhance your Action. You may have noticed that, after winning or losing a session of the Snow Pal game in the simulator, the Game over! screen includes a non-functional Play Again button. Your challenge is to configure this Play Again button to begin a new game session when the user clicks it.

Keep the following information in mind:

If you need help, see Part 6: Configure the Start Game intent and functionality. To see the code that makes the Play Again button work, see the following section.

Solution

To configure the Play Again button to start a new game, add the line between /* ADD CODE HERE...*/ and /* END */ to the create() method of start/public/js/game.js:

game.js

    this.playAgainButton = new Phaser.GameObjects.Text(this, 0, this.scale.height / 1.5, 'Play Again', {fontSize: 75, fill: '#000'});
     this.playAgainButton.x = (this.scale.width / 2) - (this.playAgainButton.width / 2);
     this.playAgainButton
     .setVisible(false)
     .setInteractive({ useHandCursor: true })
     .on('pointerover', () => this.playAgainButton.setStyle({fill: '#ff0'}))
     .on('pointerout', () => this.playAgainButton.setStyle({fill: '#000'}))
     .on('pointerdown', () => this.playAgainButton.setStyle({fill: '#0ff'}))
     .on('pointerup', () => {
         this.playAgainButton.setStyle({fill: '#ff0'});
         /* ADD CODE HERE to use sendTextQuery() for 'Play again' button: */
          window.interactiveCanvas.sendTextQuery('Play again');
         /* END */
     });
     this.add.existing(this.playAgainButton);

Here, you use the sendTextQuery() method to programmatically invoke the Play Again intent when the user clicks the Play Again button. To trigger the intent, you pass in one of its training phrases, Play again. The intent is matched and updates the web app to display a new game session.

Make sure you redeploy your fulfillment and web app files after editing the code!

Test the Play Again button

To test the Play Again button, follow these steps:

  1. Navigate to the simulator in the Actions console and play the Snow Pal game.
  2. At the conclusion of the game, click the Play Again button. A new game session should begin in the simulator.

In this section, you learn how to debug your Interactive Canvas Action when it's not working properly. The Snow Pal project comes pre-packaged with a debugging overlay that is enabled by default. The overlay displays all console.log() and console.error() output to the bottom right of the display, as shown in the following screenshot:

To disable this overlay, open the /public/css/main.css file and uncomment the line /* display: none !important; */ :

main.css

.debug {
 display: flex;
 flex-direction: column;

 /* Uncomment below to disable the debug overlay */
 /* display: none !important; */

 width: 500px;
 height: 150px;
 right: 0;
 bottom: 0;
 position: absolute;
}

To add the debugging overlay to your own Interactive Canvas Action, follow these steps:

  1. Add <div> tags to the <body> of your main index.html file to update your project with a logging overlay:

index.html

<div class="view">
   <div class="debug">
     <div class="logs"></div>
   </div>
 </div>
  1. Link the following files to your index.html file:

Congratulations!

You've completed the introductory Interactive Canvas codelab, and now have the skills necessary to build your own Interactive Canvas gaming Action.

What you've learned

Additional learning resources

Check out the following resources to learn more about Interactive Canvas: