A Tour of Cloud IoT Core

1. Introduction

Cloud IoT Core is a fully managed service that allows you to easily and securely connect, manage, and ingest data from millions of globally dispersed devices. Cloud IoT Core, in combination with other services on Google Cloud platform, provides a complete solution for collecting, processing, analyzing, and visualizing IoT data in real time to support improved operational efficiency.

This chart shows how the Google Cloud IoT Core bridge connects devices to Google Cloud Services. It also highlights the data and analytics tools for warehousing and inspecting your data.

This codelab gives you a hands-on walkthrough to setting up and configuring IoT devices using Cloud IoT Core. You will transmit telemetry messages from a device and the device will respond to configuration changes from a server based on real-time data.

What you'll build

In this codelab, you'll build a simple but complete IoT system. The devices in this system publish temperature data to their telemetry feeds, and a server consumes the telemetry data from a Cloud Pub/Sub topic. The server then decides whether to turn on or off the individual devices' fans, via a Cloud IoT Core configuration update.

The diagram shows four components: a device, Cloud IoT, a server, and Cloud Pub/Sub.  When the device publishes temperature data, Cloud IoT sends it to Cloud Pub/Sub.   The example server consumes the Pub/Sub messages with the temperature data and then sends commands to Cloud IoT Core if the temperature is outside the desired range.  Cloud IoT Core transmits the command back to the device.

What you'll learn

  • How to create a device registry and add a device to it
  • How to provision a device and transmit telemetry data from it
  • How to control a device using a server based on a telemetry stream
  • [Optional] How to go from virtual device to hardware

What you'll need

This codelab focuses on Google Cloud IoT Core. Tangential concepts and code blocks are glossed over and are provided for you to simply copy and paste.

2. Use Cloud IoT Core with Virtual Devices

You will connect a virtual device and run the server that listens to the telemetry messages from the connected device, then use the server to control that virtual device.

3. Start in the Google Cloud Console

Click the following link to open the end-to-end sample in Google Cloud Shell:

If you have already cloned the repo, input 1 to cd into the sample folder and then cd iot/api-client/end_to_end_example, otherwise enter the following command to cd into the sample folder:

cd python-docs-samples/iot/api-client/end_to_end_example

Next, initialize virtual environment and install the sample dependencies:

virtualenv env && source env/bin/activate
pip install -r requirements.txt

At this point, you can check that you have installed the Python dependencies correctly by running all the Python scripts without passing any parameters:

python cloudiot_pubsub_example_mqtt_device.py
python cloudiot_pubsub_example_server.py

If the dependencies installed successfully, the programs will print their respective usage messages, for example:

usage: cloudiot_pubsub_example_server.py [-h] --project_id PROJECT_ID
                                         [--service_account_json SERVICE_ACCOUNT_JSON]

usage: cloudiot_pubsub_example_mqtt_device.py [-h] --project_id PROJECT_ID
                                              --registry_id REGISTRY_ID
                                              --device_id DEVICE_ID
                                              PRIVATE_KEY_FILE --algorithm
                                              [--cloud_region CLOUD_REGION]
                                              [--ca_certs CA_CERTS]
                                              [--num_messages NUM_MESSAGES]
                                              [--mqtt_bridge_hostname MQTT_BRIDGE_HOSTNAME]
                                              [--mqtt_bridge_port MQTT_BRIDGE_PORT]
                                              [--message_type {event,state}]

If you see an error similar to ImportError: No module named ..., go back and make sure you installed Virtual Environment correctly, or see the Python Development Environment Setup Guide for detailed information on using Python with Google Cloud. Now that you have the program libraries installed, it's time to set up your Google Cloud IoT Core project.

4. Set up the virtual device demo

To ensure your Cloud shell has the latest versions of the Cloud IoT Core API installed, update the gcloud components.

gcloud components update

Open the Google Cloud Console and enable the API if it has not already been enabled on your Kiosk or Google account.

After you have enabled the API, select the current project to get your project ID.

The selection menu to the right of the Google Cloud Logo is highlighted showing the menu item to click for project selection.

The project name shown in the menu bar can differ from the project ID, so make sure you set it to the correct value.

gcloud config set project <your-project-id>

Create a Pub/Sub topic using the following gcloud command:

gcloud pubsub topics create tour-pub --project="${DEVSHELL_PROJECT_ID:-Cloud Shell}"
gcloud pubsub subscriptions create tour-sub --topic=tour-pub

Create your Cloud IoT Device Registry using the following gcloud command:

gcloud iot registries create tour-registry \
  --region=us-central1 --event-notification-config=topic=tour-pub

Next, you will need to generate RSA public and private keys that will be used for authenticating your virtual device when it connects.

openssl req -x509 -newkey rsa:2048 -days 3650 -keyout rsa_private.pem \
    -nodes -out rsa_public.pem -subj "/CN=unused"

With your keys in hand, you're ready to register a device. Register your device using the public key.

gcloud iot devices create test-dev --region=us-central1 \
  --registry=tour-registry \
  --public-key path=rsa_public.pem,type=rs256

Congratulations, you have now set up Cloud Pub/Sub, created your device registry, and added a device to the registry!

5. Start the virtual server

Now that you have created all of the Cloud resources you need to connect your device and communicate with it via Pub/Sub, it's time to simulate a device and server.

Edit cloudiot_pubsub_example_server.py and replace the code used to generate service account credentials from a provided JSON file to instead use the built-in credentials for Compute Engine, which is what is running under the hood of the Cloud Shell.

Replace the following code:

    def __init__(self, service_account_json):
        credentials = ServiceAccountCredentials.from_json_keyfile_name(
            service_account_json, API_SCOPES)


    def __init__(self, service_account_json):        
        from google.auth import compute_engine
        credentials = compute_engine.Credentials()

Now that you have changed the server to use the Compute Engine credentials, start the server using the following syntax:

python cloudiot_pubsub_example_server.py \
    --project_id="PROJECT_ID" \

For example, if you used the example values in previous commands, the command is:

python cloudiot_pubsub_example_server.py \
    --project_id="${DEVSHELL_PROJECT_ID:-Cloud Shell}" \

When the server starts, you will see the message "Listening for messages on projects/your-project-id/subscriptions/tour,"which indicates the server is running.

6. Start the virtual device and observe

Add a new tab to your Cloud Shell by clicking the + icon on the menu bar. Navigate to the device sample folder and initialize your virtual environment using the following commands:

cd python-docs-samples/iot/api-client/end_to_end_example
source env/bin/activate

Retrieve the latest root certificate from Google:

wget pki.goog/roots.pem

Now, connect the virtual device using the private key, registry ID, device ID, and so on:

python cloudiot_pubsub_example_mqtt_device.py \
  --registry_id tour-registry \
  --device_id test-dev \
  --project_id "${DEVSHELL_PROJECT_ID:-Cloud Shell}" \
  --private_key_file rsa_private.pem \
  --algorithm RS256 \
  --ca_certs roots.pem \
  --cloud_region us-central1

When you connect the device, it will show and report its temperature, which increases when the fan is turned off. If the fan is enabled, the virtual device's temperature will decrease. Because the device is controlled from the server, which is analyzing the stream of incoming sensor data and making this decision for it, the device does not need to be aware of the conditions for enabling or disabling its fan.

The following section shows the output of the server that is subscribed to the telemetry events from the device.

The device (test-dev) has a temperature of: 11
('Setting fan state for device', u'test-dev', 'to on.')
The device (test-dev) has a temperature of: 10
The device (test-dev) has a temperature of: 9
The device (test-dev) has a temperature of: 8
The device (test-dev) has a temperature of: 7
The device (test-dev) has a temperature of: 6
The device (test-dev) has a temperature of: 5
The device (test-dev) has a temperature of: 4
The device (test-dev) has a temperature of: 3
The device (test-dev) has a temperature of: 2
The device (test-dev) has a temperature of: 1
The device (test-dev) has a temperature of: 0
The device (test-dev) has a temperature of: -1
('Setting fan state for device', u'test-dev', 'to off.')
The device (test-dev) has a temperature of: 0
The device (test-dev) has a temperature of: 1
The device (test-dev) has a temperature of: 2

The following section shows the output of the device that is transmitting its telemetry events to the server.

('Publishing payload', '{"temperature": 11}')
Published message acked.
Received message '{"fan_on": true}' on topic '/devices/test-dev/config' with Qos 1
Fan turned on.
('Publishing payload', '{"temperature": 10}')
Published message acked.
('Publishing payload', '{"temperature": 9}')
Published message acked.
('Publishing payload', '{"temperature": 8}')
Published message acked.
('Publishing payload', '{"temperature": 7}')
Published message acked.
('Publishing payload', '{"temperature": 6}')
Published message acked.
('Publishing payload', '{"temperature": 5}')
Published message acked.
('Publishing payload', '{"temperature": 4}')
Published message acked.
('Publishing payload', '{"temperature": 3}')
Published message acked.
('Publishing payload', '{"temperature": 2}')
Published message acked.
('Publishing payload', '{"temperature": 1}')
Published message acked.
('Publishing payload', '{"temperature": 0}')
Published message acked.
('Publishing payload', '{"temperature": -1}')
Published message acked.
Received message '{"fan_on": false}' on topic '/devices/test-dev/config' with Qos 1
Fan turned off.
('Publishing payload', '{"temperature": 0}')
Published message acked.
('Publishing payload', '{"temperature": 1}')
Published message acked.
('Publishing payload', '{"temperature": 2}')

You have now set up a virtual device and are successfully transmitting telemetry data and receiving configuration changes! At this point you have a basic understanding of Google Cloud IoT Core.

Cleaning up

After you're finished with the codelab, run the following commands to clean up the resources you created on your Cloud account:

gcloud iot devices delete test-dev --region=us-central1 --registry=tour-registry
gcloud iot registries delete tour-registry --region=us-central1
gcloud pubsub subscriptions delete tour-sub
gcloud pubsub topics delete tour-pub