Internet of Things (IoT) developers can integrate their devices with the Google Assistant by building smart home Actions. With a smart home Action, your users can control their devices through voice commands, as well as using touch controls in the Google Home app.

Smart home Actions rely on Home Graph, a database that stores and provides contextual data about the home and its devices. For example, Home Graph can store the concept of a living room that contains multiple types of devices (such as a thermostat, lamp, fan, and vacuum) from different manufacturers. This information is passed to the Google Assistant in order to execute user requests based on the appropriate context.

What you'll build

In this codelab, you'll publish a cloud service that contains a virtual smart washing machine. You will then go through the process of connecting that service to the Google Assistant with a smart home Action.

What you'll learn

What you'll need

Enabling Activity controls

Enable the following Activity controls in the Google Account you plan to use with the Assistant:

Using the Actions on Google console

  1. Go to the Actions on Google Developer Console.
  2. Click New Project, enter a name for the project, and click CREATE PROJECT.

Select the Smart Home App

On the Overview screen in the Actions console, select Smart home.

Choose the Smart home experience card, and you will then be directed to your project console.

Install the Firebase Command Line Interface

The Firebase Command Line Interface (CLI) will allow you to serve your web apps locally and deploy your web app to Firebase hosting.

To install the CLI, run the following npm command from the terminal:

npm install -g firebase-tools

To verify that the CLI has been installed correctly, run:

firebase --version

Authorize the Firebase CLI with your Google account by running:

firebase login

Now that you've set up your development environment, let's deploy the starter project to verify everything is configured properly.

Get the source code

Click the following link to download the sample for this codelab on your development machine:

Download source code

...or you can clone the GitHub repository from the command line:

git clone https://github.com/googlecodelabs/smarthome-washer.git

About the project

The starter project contains the following subdirectories:

The project contains a fully implemented cloud service that manages a virtual smart washer device using Firebase and a web frontend to view and control device state. The hooks to extend that control to the Google Assistant are currently empty stubs.

Connect to Firebase

Navigate to the washer-start directory then set up the Firebase CLI with your Actions Project:

cd washer-start
firebase use <project-id>

Deploy to Firebase

Navigate to the functions folder and install all the necessary dependencies using npm.

cd functions
npm install

Now that you have installed the dependencies and configured your project, you are ready to run the app for the first time.

firebase deploy

This is the console output you should see:

...

✔ Deploy complete!

Project Console: https://console.firebase.google.com/project/<project-id>/overview
Hosting URL: https://<project-id>.firebaseapp.com

This command deploys a web app, along with several Cloud Functions for Firebase.

Open the Hosting URL in your browser (https://<project-id>.firebaseapp.com) to view the web app. You will see the following interface:

This web UI represents a third-party platform to view or modify device states. To begin populating your database with device information, click UPDATE. You won't see any changes on the page, but the current state of your washer will be stored in the database.

Now it's time to connect the cloud service you've deployed to the Google Assistant using the Actions console.

Configure your project in the Actions console

Under Overview > Build your Action, select Add Action(s). Enter the URL for your cloud function that provides fulfillment for the smart home intents and click Save.

https://us-central1-<project-id>.cloudfunctions.net/smarthome

On the Develop > Invocation tab, add a Display Name for your Action, and click Save. This name will appear in the Google Home app.

To enable Account linking, select the Develop > Account linking option in the left navigation. Use these account linking settings:

Client ID

ABC123

Client secret

DEF456

Authorization URL

https://us-central1-<project-id>.cloudfunctions.net/fakeauth

Token URL

https://us-central1-<project-id>.cloudfunctions.net/faketoken

Click Save to save your account linking configuration, then click Test to enable testing on your project.

You will be redirected to the Simulator. Verify that testing has been enabled for your project by moving your mouse over the Testing on Device ( ) icon.

You are now ready to begin implementing the webhooks necessary to connect the device state with the Google Assistant.

Now that your Action is configured, you can start adding devices and sending data. There are three intents that your cloud service needs to handle:

In this section, you will update the functions you previously deployed to handle these intents.

Update SYNC response

Open functions/index.js. This contains the code to respond to requests from the Google Assistant.

First, we will need to handle a SYNC intent by returning the device metadata and capabilities. Update the JSON in the onSync array to include the device information and recommended traits for a clothes washer.

index.js

app.onSync((body) => {
  return {
    requestId: body.requestId,
    payload: {
      agentUserId: '123',
      devices: [{
        id: 'washer',
        type: 'action.devices.types.WASHER',
        traits: [
          'action.devices.traits.OnOff',
          'action.devices.traits.StartStop',
          'action.devices.traits.RunCycle',
        ],
        name: {
          defaultNames: ['My Washer'],
          name: 'Washer',
          nicknames: ['Washer'],
        },
        deviceInfo: {
          manufacturer: 'Acme Co',
          model: 'acme-washer',
          hwVersion: '1.0',
          swVersion: '1.0.1',
        },
        willReportState: true,
        attributes: {
          pausable: true,
        },
      }],
    },
  };
});

Deploy to Firebase

Deploy the updated cloud fulfillment using the Firebase CLI:

firebase deploy --only functions

Link to the Google Assistant

In order to test your smart home Action, you need to link your project with a Google account. This enables testing through Google Assistant surfaces and the Google Home app that are signed in to the same account.

  1. On your phone, open the Google Assistant settings. Note that you should be logged in as the same account as in the console.
  2. Navigate to Google Assistant > Settings > Home Control (under Assistant).
  3. Select the plus (+) icon in the bottom right corner
  4. You should see your test app with the [test] prefix and the display name you set.
  5. Select that item. The Google Assistant will then authenticate with your service and send a SYNC request, asking your service to provide a list of devices for the user.

Open the Google Home app and verify that you can see your washer device.

Now that your cloud service is properly reporting the washer device to Google, it's time to add the ability to request the device state and send commands.

Handling QUERY intent

A QUERY intent will include a set of devices. For each device, you should respond with its current state.

In functions/index.js, edit the QUERY handler to process the list of target devices contained in the intent request:

index.js

app.onQuery(async (body) => {
  const {requestId} = body;
  const payload = {
    devices: {},
  };
  const queryPromises = [];
  const intent = body.inputs[0];
  for (const device of intent.payload.devices) {
    const deviceId = device.id;
    queryPromises.push(queryDevice(deviceId)
      .then((data) => {
        // Add response to device payload
        payload.devices[deviceId] = data;
      }
    ));
  }
  // Wait for all promises to resolve
  await Promise.all(queryPromises)
  return {
    requestId: requestId,
    payload: payload,
  };
});

For each device contained in the request, return the current state stored in the Firebase database. Update the queryFirebase and queryDevice functions to return the state data of the washer.

index.js

const queryFirebase = async (deviceId) => {
  const snapshot = await firebaseRef.child(deviceId).once('value');
  const snapshotVal = snapshot.val();
  return {
    on: snapshotVal.OnOff.on,
    isPaused: snapshotVal.StartStop.isPaused,
    isRunning: snapshotVal.StartStop.isRunning,
  };
}

const queryDevice = async (deviceId) => {
  const data = await queryFirebase(deviceId);
  return {
    on: data.on,
    isPaused: data.isPaused,
    isRunning: data.isRunning,
    currentRunCycle: [{
      currentCycle: 'rinse',
      nextCycle: 'spin',
      lang: 'en',
    }],
    currentTotalRemainingTime: 1212,
    currentCycleRemainingTime: 301,
  };
}

Handling EXECUTE intent

The EXECUTE intent handles commands to update device state. The response returns the status of each command (for example, SUCCESS, ERROR, PENDING) and the new device state.

In functions/index.js, edit the EXECUTE handler to process the list of traits to be updated and the set of target devices for each command:

index.js

app.onExecute(async (body) => {
  const {requestId} = body;
  // Execution results are grouped by status
  const result = {
    ids: [],
    status: 'SUCCESS',
    states: {
      online: true,
    },
  };
  const executePromises = [];
  const intent = body.inputs[0];
  for (const command of intent.payload.commands) {
    for (const device of command.devices) {
      for (const execution of command.execution) {
        executePromises.push(
          updateDevice(execution,device.id)
            .then((data) => {
              result.ids.push(device.id);
              Object.assign(result.states, data);
            })
            .catch(() => console.error(`Unable to update ${device.id}`))
        );
      }
    }
  }
  await Promise.all(executePromises)
  return {
    requestId: requestId,
    payload: {
      commands: [result],
    },
  };
});

For each command and target device, update the values in the Firebase database that correspond to the requested trait. Modify the updateDevice function to update the appropriate Firebase reference and return the updated device state.

index.js

const updateDevice = async (execution,deviceId) => {
  const {params,command} = execution;
  let state, ref;
  switch (command) {
    case 'action.devices.commands.OnOff':
      state = {on: params.on};
      ref = firebaseRef.child(deviceId).child('OnOff');
      break;
    case 'action.devices.commands.StartStop':
      state = {isRunning: params.start};
      ref = firebaseRef.child(deviceId).child('StartStop');
      break;
    case 'action.devices.commands.PauseUnpause':
      state = {isPaused: params.pause};
      ref = firebaseRef.child(deviceId).child('StartStop');
      break;
  }
  return ref.update(state)
    .then(() => state);
};

Now that you have implemented all three intents, you can test that your action controls your washer.

Deploy to Firebase

Deploy the updated cloud fulfillment using the Firebase CLI:

firebase deploy --only functions

Test the Washer

Now you can see the value change when you give a voice command. You can use your phone to give these commands.

"Turn on my washer"

"Pause my washer"

"Stop my washer"

You can also see the current state of your washer by asking questions.

"Is my washer on?"

"Is my washer running?"

"What cycle is my washer on?"

You can also see these queries and commands in your Firebase logs. You can access these from the left navigation menu in the console by clicking Develop > Functions > Logs.

Your cloud service is now fully integrated with the smart home intents, enabling users to control and query the current state of their devices. However, this implementation still lacks a way for your service to proactively send event information (such as changes to device presence or state) to the Google Assistant.

With the Request Sync API, you can trigger a new SYNC request when users add or remove devices, or when their device capabilities change. With the Report State API, your cloud service can proactively send a device's state to the Home Graph. This is helpful if the user physically changed the device state (for example, turning on a light switch) or changed the state using another service.

In this section, you will add code to call these APIs from the frontend web app.

Enable the HomeGraph API

The HomeGraph API enables the storage and querying of devices and their states within a user's Home Graph. To use this API, you must first open the Google Cloud console and enable the HomeGraph API.

In the Google Cloud console, make sure to select the project that matches your Actions <project-id>. Then, in the API Library screen for the HomeGraph API, click Enable.

Enable Report State

The reportstate function in the starter project is triggered by writes to the Firebase database. Update the reportstate function in functions/index.js to capture the data written to the database and post it to the Report State API.

index.js

exports.reportstate =
    functions.database.ref('{deviceId}').onWrite(async (change, context) => {
  console.info('Firebase write event triggered this cloud function');
  const snapshot = change.after.val();

  const requestBody = {
    requestId: 'ff36a3cc', /* Any unique ID */
    agentUserId: '123', /* Hardcoded user ID */
    payload: {
      devices: {
        states: {
          /* Report the current state of our washer */
          [context.params.deviceId]: {
            on: snapshot.OnOff.on,
            isPaused: snapshot.StartStop.isPaused,
            isRunning: snapshot.StartStop.isRunning,
          },
        },
      },
    },
  };

  const res = await homegraph.devices.reportStateAndNotification({
    requestBody
  });
  console.info('Report state response:', res.status, res.data);

});

Enable Request Sync

The requestsync function in the starter project is triggered by the refresh icon in the frontend web UI. Implement the requestsync function in functions/index.js to call the HomeGraph API.

index.js

exports.requestsync =
    functions.https.onRequest(async (request, response) => {
  response.set('Access-Control-Allow-Origin', '*');
  console.info('Request SYNC for user 123');
  try {
    const res = await homegraph.devices.requestSync({
      requestBody: {
        agentUserId: '123'
      }
    });
    console.info('Request sync response:', res.status, res.data);
    response.json(res.data);
  } catch (err) {
    console.error(err);
    response.status(500).send(`Error requesting sync: ${err}`)
  }

});

Deploy to Firebase

Deploy the updated code using the Firebase CLI:

firebase deploy --only functions

Test your implementation

Click the refresh icon in the frontend web UI, and verify that you see a sync request in the Firebase console log.

Next, adjust the attributes of the washer device in the frontend web UI and click UPDATE. Verify that you can see the state change reported to Google in your Firebase console logs.

Congratulations! You've successfully integrated Google Assistant with a device cloud service using smart home Actions. Here are some ideas you can implement to go deeper:

You can also learn more about testing and submitting an Action for review, including the certification process to publish your Action to users.

What we've covered