google-iot

Google IoT meets ESP32, MQTT, Node JS

Posted on

ESP32, MQTT, NodeJS and Google IoT

This tutorial gives you a glimpse on integrating IoT devices with Google’s ecosystem. This approach is scalable to a great many devices, has security designed in an only relies on the IoT device and Google Cloud Platform.

This approach is more involved compared to our earlier approaches [1,2] that relied on many in between systems. This time around, the only programming will be in JavaScript (Node JS). No opening ports on the firewall, no IFT3. Plain and simple, ‘though never as simple as getting as opening the door to see how the weather is.

This guide was inspired by Alvaro Viebrantz’ post [3]. With MongooseOS and Google Cloud IoT still in beta, it occasionally felt like the ground was moving under my feet, but due patience and persistence makes engineering great. With this write-up, I try to make it less daunting while providing an understanding of the inner workings.

This write-up is structured as a step-by-step guide to help you understand the elements. If you prefer to find your own way around, feel free to head over to the GitHub repository (no longer life!). We use Windows 10, but the instructions apply to other operating systems as well. Just note that in Windows PowerShell is the character ` is used to split up long lines.

Design

Goal

The design needs to be scalable, secure and fit for low-power SoC applications.

The lineup

Block diagram: IoT integration with Google Cloud
IoT integration with Google Cloud

Message Protocol

The key ingredient is the MQTT protocol. The abbreviation MQTT stands for Message Queue Telemetry Transport, where the keyword is telemetry, or remote monitoring. Its publish/subscription model makes it an efficient protocol that is well suited for low power devices. By creating topics, you can enable different parts of your application to subscribe to specific streams of data. MQTT runs on the TCP transport that provides reliable delivery, but also requires devices to be awake at least some percentage of the time. The MQTT protocol is an efficient binary industry standard that allows constrained devices to send real-time telemetry as well as immediately receive messages sent from the cloud. I designed wearable products that used a phone to bridge from BTLE to MQTT in large scale deployment on AWS IoT. This time around, I will keep things as simple as possible while still keeping the design criteria front and center.

The “thing”

To keep things simple we an Espressif ESP32 System-on-a-Chip (SoC) that has builtin WiFi support. It would not be my first choice for low power, but it is cheap and is well supported. We connect a simple temperature and humidity sensor to generate some data. Feel free to use whatever other sensor comes to mind.

Google Cloud Platform

On the backend, we use the Google Cloud Platform that recently introduced MQTT support as “Cloud IoT”. The SoC will publish its readings to Cloud IoT, from where they are stored. For visualization we use the Data Studio tool. The block diagram at the beginning of this chapter shows the data flow. [4]

Cloud IoT Core

IoT Core handles registration, authentication and authorization inside the Cloud Platform resource hierarchy. It connects to Cloud Pub/Sub to provide a secure MQTT broker service. It also provides the ability to send device configuration to devices.

Cloud Pub/Sub

Google Cloud Pub/Sub implements MQTT by providing a globally durable message ingestion service. Cloud Pub/Sub can act like a shock absorber and rate leveler for both incoming data streams and application architecture changes. Many devices have limited ability to store and retry sending telemetry data. Cloud Pub/Sub scales to handle data spikes that can occur when swarms of devices respond to events in the physical world, and buffers these spikes to help isolate them from applications monitoring the data. [src]

Firebase real-time database

Firebase Realtime Database is a cloud-hosted NoSQL database that stores data as JSON. We use it to store the last value read from the sensors.

Cloud BigQuery

Cloud BigQuery enables standard SQL queries for interactive analysis of massively large datasets working in conjunction with Google Storage.

Firebase Functions

Firebase Functions lets you run backend code that responds to events in the Cloud. We will have it responds to data published to a Pub/Sub topic, or a request from DialogFlow to a HTTPS endpoint.

Data Studio

Data Studio can visualize data through configurable charts and tables. We are going to use it to visualize the historic data stored in BigQuery.

Device to Cloud Pub/Sub

Block diagram: Device to Cloud Pub/Sub
Block diagram: Device to Cloud Pub/Sub

Google Cloud

We use Google Cloud to store and access the readings from our IoT device. To get started, head over Cloud Console and create a project and prepare it for Message Queuing Telemetry Transport (MQTT). MQTT is a lightweight machine-to-machine publish/subscribe protocol, often used where a small code footprint is required and/or network bandwidth is at a premium.

    • + to create a project with a unique name (this post assumes the name “esp-nodejs-mqtt”)
    • Enable billing. Google Cloud has a generous free quota, but read the pricing and terms to ensure you are comfortable with them.
    • enable “ Google Cloud IoT”
    • enable “ Google Cloud Pub/Sub”
  4. Connect your services with reliable, many-to-many, asynchronous messaging hosted on Google’s infrastructure.
    • Name: mqtt-topic
  5. A device registry allows you to group and set properties that they all share, such as connection protocol, data storage location, and CLoud Pub/Sub topics.
    • Registry ID: mqtt-registry
    • Cloud region: us-central1 (whatever is closer to you)
    • Protocol: MQTT, HTTP
    • Default telemetry topic: mqtt-topic

We will add the device and its public key in the next section.

The “thing”

Our “thing” will be an Espressif ESP32 board with an integrated USB-serial, such as the Adafruit HUZZAH32. No particular reason, besides that it is cheap and has build-in WiFi. A temperature and humidity sensor is connected to GPIO21. If you use an ESP8266, connect the sensor to GPIO4.

On the ESP32, we use a off-the-shelf firmware that can be programmed using JavaScript. The firmware, called Mongoose OS, comes with a library to interface with Google IoT Core. [github, github].

Tools

The command line tool mos generates custom firmware and uploads it to the ESP32. The tool also gives easy access to the serial debug port and SPIFFS file system. For integration with Cloud IoT, it relies on Google’s gcloud command line tool.

gcloud

We will start with installing the Google Cloud tool and linking it to our project.

  1. Open a command terminal, create a project directory and make that the active directory.
  2. Install the gcloud command line tool as described in Cloud SDK.
    • Choose to include all components (incl. Beta components).
    • Accept all post-install options. Once it executes gcloud init, choose to
      • login (to Google), and follow the directions,
      • pick your cloud project (for us that was “esp-nodejs-mqtt”).
      • It should finish with “Your Google Cloud SDK is configured and ready to use”
  3. Give Cloud IoT access your project 
    gcloud projects add-iam-policy-binding esp-nodejs-mqtt `
        --member=serviceAccount:cloud-iot@system.gserviceaccount.com `
        --role=roles/pubsub.publisher
    Remember to use your own project id (see cloud projects list).

mos

Install the Mongoose OS tool and install the Cloud IoT’s CA certificate

  1. Open a command terminal, and make your project directory the current directory.
  2. Start with some boilerplate code (install git if you haven’t done so already) 
    git clone https://github.com/mongoose-os-apps/empty firmware
  3. Install the mos tool in the firmware directory according to its instructions.
  4. Downgrade to version 1.22 because this seems to be the last stable version 
    firmware\mos update 1.22
  5. Connect the ESP32 to your computer using USB.
  6. Make the firmware you current directory.
  7. Retrieve Google Cloud’s CA certificate using a *nix system with openssl installed 
    echo -n | openssl s_client -showcerts -connect mqtt.googleapis.com:8883 | \
        sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > ca.pem
    Copy this ca.pem to firmware\fs\ca.pem

Program the device

With Cloud IoT setup, it is time to tell the device what to do. For this tutorial, the device will measure the temperature and humidity every 5 seconds. Every 15 minutes it will publish the readings to Cloud IoT as topic /devices/DEVICENAME/events/.

  1. Update the build description firmware\mos.yml with the apps’ configuration and libraries 
    author: Alvaro Viebrantz, modified by Coert Vonk
description: esp-nodejs-iot firmware
version: 1.0

libs_version: ${mos.version}
modules_version: ${mos.version}
mongoose_os_version: ${mos.version}

# List of files/directories with C sources. No slashes at the end of dir names
sources:
    - src

# Files from these dirs will be copied to the device filesystem
filesystem:
    - fs

# Custom configuration entries, settable via "device configuration"
config_schema:
    - ["i2c.enable", true]
    - ["app", "o", { title: "App custom settings"}]
    - ["app.dht", "i", 21, { title: "DHT pin"}]

# List of libraries used by this app, in order of initialisation
libs:
# use fs\ca.pem instead of https://github.com/mongoose-os-libs/ca-bundle
# - origin: https://github.com/mongoose-os-libs/ca-bundle
    - origin: https://github.com/mongoose-os-libs/rpc-service-config
    - origin: https://github.com/mongoose-os-libs/rpc-service-fs
    - origin: https://github.com/mongoose-os-libs/rpc-uart
    - origin: https://github.com/mongoose-os-libs/wifi
# libs for the current app
    - origin: https://github.com/mongoose-os-libs/mqtt
    - origin: https://github.com/mongoose-os-libs/i2c
    - origin: https://github.com/mongoose-os-libs/gcp
    - origin: https://github.com/mongoose-os-libs/mjs
    - origin: https://github.com/mongoose-os-libs/dht

# Used by the mos tool to catch mos binaries incompatible with this file format
manifest_version: 2017-05-18
  2. When using the ESP8266, add a device-type specific build description firmware\mos_esp8266.yml to change the DHT pin to GPIO4 
    # ESP8266 specific configuration entries
config_schema:
    - ["app.dht", "i", 4, { title: "DHT pin"}]
  3. Create a JavaScript program firmware\fs\init.js to read the sensor and periodically publish it to Cloud IoT. The program assigns the sensor pin based on the app.dht property in the yml build description. 
    load('api_config.js');
load('api_mqtt.js');
load('api_timer.js');
load('api_dht.js');
load('api_config.js');
load('api_gpio.js');
load('api_mqtt.js');
load('api_net.js');
load('api_sys.js');

let appName = 'init.js             ';
let deviceName = Cfg.get('device.id');

// The MQTT topic name is required to be in the format below. The topic name must end
// in 'state' to publish state and 'events' to publish telemetry. Note that this is
// not the same as the device registry's Cloud Pub/Sub topic.
let topic = '/devices/' + deviceName + '/events';
print(appName, 'Topic:', topic);

let isConnected = false;
let dhtPin = Cfg.get('app.dht');
let dht = DHT.create(dhtPin, DHT.DHT11);

function getInfo() {
    return JSON.stringify({
        total_ram: Sys.total_ram() / 1024,
        free_ram: Sys.free_ram() / 1024,
        temp: dht.getTemp(),
        hum: dht.getHumidity()
    });
};

function publishData() {
    if (MQTT.pub(topic, getInfo())) {
        print(appName, 'Published');
    }
}

Timer.set(
    15 * 60 * 1000, true,
    function() {
        if (isConnected) {
            print(appName, '5 minute timer, CONNECTED > PUBLISH');
            publishData();
        }
    }, null
);

Timer.set(
    5 * 1000, true,
    function() {
        print(appName, 'Info:', getInfo());
    }, null
);

MQTT.setEventHandler(function(conn, ev, data) {
    if (ev !== 0) {
        let evs = '?';
        if (ev === MQTT.EV_CONNACK) {
            evs = 'CONNACK';   // Connection to broker has been established
        } else if (ev === MQTT.EV_PUBLISH) {
            evs = 'PUBLISH';   // msg published to topics we are subscribed to
        } else if (ev === MQTT.EV_PUBACK) {
            evs = 'PUBACK';    // ACK for publishing of a message with QoS>0
        } else if (ev === MQTT.EV_SUBACK) {
            evs = 'SUBACK';    // ACK for a subscribe request
        } else if (ev === MQTT.EV_UNSUBACK) {
            evs = 'UNSUBACK';  // ACK for an unsubscribe request
        } else if (ev === MQTT.EV_CLOSE) {
            evs = 'CLOSE';     // connection to broker was closed
        }
        print(appName, 'MQTT event:', evs, ev);
        if (ev === MQTT.EV_CONNACK) {
            print(appName, 'MQTT CONNACK > PUBLISHING');
            isConnected = true;
            publishData();
        }
    }
}, null);
  4. Create a binary and file system for the device 
    .\mos build --arch esp32
.\mos flash

Deploy

Register with Cloud IoT and configure WiFi

  1. Register the device with Cloud IoT. This single command will
    • use the device id to generate a key pair,
    • upload the private key to the device,
    • add gcp and mqtt configuration to the conf9.json file on the device,
    • (re)create the device with its public key to Cloud IoT.
    .\mos gcp-iot-setup --gcp-project esp-nodejs-mqtt --gcp-region us-central1 ^
        --gcp-registry mqtt-registry
    Once more, remember to use your own project id and geographic region. If successful, it will show the JSON gcp and mqtt configuration. If the command hangs, make sure that GPIO12 is not pulled down.
  2. Add the WiFi credentials to the configuration conf9.json file on the device 
    .\mos wifi "yourWiFiSSID" "yourWiFiPasswd"

Expected results and debugging

The device should be connecting to Cloud IoT as evident from the device’s debug trace and current value pulled from Cloud IoT

  1. This should show when the last telemetry event was received, and the device’s ES256 public key.
  2. Show the debug output of the device 
    .\mos console

      3. If things go well you will see debug messages from the device such as 

      esp32_mgos_init      firmware 1.0 (20180309-003629/???)
mgos_i2c_create      I2C GPIO init ok (SDA: 32, SCL: 33)
init.js              Topic:  /devices/esp32_F00F00/events 
init.js              Net event: CONNECTING 1 
init.js              Net event: CONNECTED 2 
init.js              Net event: GOT_IP 3 
mgos_mqtt_global_con MQTT connecting to mqtt.googleapis.com:8883
mgos_sntp_query      SNTP query to time.google.com
mgos_mqtt_ev         MQTT TCP connect ok (0)
mgos_sntp_ev         SNTP reply from 216.239.35.4: time 1520555855.896951, loc...
init.js              Info: {"hum":55,"temp":21,"free_ram":201.261719,"total_ra... 
mgos_mqtt_ev         MQTT CONNACK 0
init.js              MQTT event: CONNACK 202 
init.js              MQTT CONNACK > PUBLISHING 
init.js              Published 
init.js              Info: {"hum":48,"temp":21,"free_ram":202.527344,"total_ra... 
init.js              Info: {"hum":57,"temp":21,"free_ram":202.527344,"total_ra,..
      Debug messages from the ESP32 device

      If things don’t go so smooth, you’re best starting point is using the MongooseOS web interface (.\mos ui). A common cause of errors is uploading the firmware after configuring the device causing the configuration to be overwritten. Net connection errors may be the effect of missing or incorrect WiFi credentials in conf9.json on the device. If you encounter MQTT CONNACK 4 messages, check the ca.pem certificate. This should match the certificate pulled using openssl earlier. Also try switching to version 1.22 of the mos tool if aren’t using this already. CONNACK 5 on the other hand, hints at using the wrong registry or project name.

      Cloud Pub/Sub to BigQuery

      We use Cloud BigQuery to store the device data. Google BigQuery is a web service that lets you do interactive analysis of massive datasets—up to billions of rows.

      Block diagram: Cloud Pub/Sub to BigQuery and Firebase Store

      BigQuery storage

      To get started, head over to BigQuery Console and create a dataset and table.

        • Dataset ID: esp_nodejs_mqtt (your project name)
        • Data location: US (whatever is closer)
        • Data expiration: Never
        • Source Data: Create empty table
        • Destination table name: esp_node_js.raw_data
        • Schema fields: 
          deviceId  STRING     REQUIRED
    timestamp TIMESTAMP  REQUIRED
    temp      FLOAT      REQUIRED
    humidity  FLOAT      REQUIRED
        • Partitioning type = Day

      Firebase functions

      Firebase provides easy access to Google Cloud services intended for mobile or web.

      Link Firebase to your Cloud project using the Firebase Console

        • Project name: esp-node-js-mqtt (your project name from the pull down list)
        • Add Firebase Confirm plan

      node tool

      Start with installing Node JavaScript (NodeJS).

      1. Open a command terminal and make your project directory the current directory. Create a subdirectory (e.g. firebase and make that the current directory.
        • Firebase doesn’t tolerate spaces in the file path. On Windows, you can work around this creating a symbolic link (mklink /d) or map a virtual drive to a UNC path (net use).
      2. Install the LTS version of NodeJS from nodejs.org with the default options.
      3. Update the NodeJS package manager from a command terminal 
        npm i npm

      firebase tool

      1. Install the NodeJS package “firebase”. We install the bleeding edge to work around issue #610. The -g parameter causes the modules to be installed globally (in %APPDATA%\npm). 
        npm install -g git://github.com/firebase/firebase-tools#master
        The install may warn that node-uuid has been superseded with uuid.
      2. Sign in to your Google account using Firebase 
        firebase login
      3. Initialize the project directory 
        firebase init
        Make the following selections
        • Project setup
          • install all except FireStore,
          • select your project as the default Firebase project (in our case esp-nodejs-mqtt).
        • Database setup
          • database rules: database.rules.json.
        • Functions setup
          • language to use: JavaScript,
          • use ESLint: yes,
          • install dependencies with npm: yes.
          • This creates the files functions\package.json, functions\.eslintrc.json and functions\index.js.
        • Hosting setup
          • public directory: public,
          • rewrite all urls to /index.html: no.
          • This creates the files public/404.html and public/index.html.
        • Storage setup
          • file for storage rules: storage.rules.
        • It finishes with creating the configuration file firebase.json and project info .firebaserc.

      Program the functionality

      Time to write some software! Configure and setup a listener in Firebase to store the MQTT messages in BigQuery, and maintain the device’s state in Firebase realtime store. This gives the cloud instant access event when the device is in low-power sleep mode and/or disconnected.

      1. Point Firebase to the BigQuery table 
        firebase functions:config:set ^
bigquery.datasetname="esp_nodejs_mqtt" ^
bigquery.tablename="history"
      2. Update the file functions\index.js (in your firebase directory). This will
        • rxMqtt() receives data from Cloud Pub/Sub, forward it to Cloud BigQuery and stores the last value in Firebase realtime storage.
        const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);

const bigquery = require('@google-cloud/bigquery')();
const db = admin.database();
const cors = require('cors')({ origin: true });

// MQTT to BigQuery and maintain current state in Firebase

function updateCurrentStatusInFirebase(data) {
    return db.ref(`/devices/${data.deviceId}`).set({
        humidity: data.humidity,
        temp: data.temp,
        timestamp: data.timestamp
    });
}
    
function insertIntoBigQuery(data) {
    const dataset = bigquery.dataset(functions.config().bigquery.datasetname);
    const table = dataset.table(functions.config().bigquery.tablename);
    return table.insert(data);
}

exports.rxMqtt = functions.pubsub
    .topic('mqtt-topic')
    .onPublish(event => {
    console.log('received topic');
        const attributes = event.data.attributes;
        const message = event.data.json;
        const deviceId = attributes['deviceId'];
        const data = {
            humidity: message.hum,
            temp: message.temp,
            deviceId: deviceId,
            timestamp: event.timestamp
        };
        if ( message.hum < 0 || message.hum > 100 ||
                message.temp > 100 || message.temp < -50 ) {
            return null;  // ignore false readings
        }
        return Promise.all(&#91;
            insertIntoBigQuery(data),
            updateCurrentStatusInFirebase(data)
        &#93;);
    });&#91;/code&#93;
        </li>
        <li>
            Install the modules referenced by <code>functions\package.json</code>.

            cd functions
npm install
cd ..

      Deploy

      The moment of truth is here: time to deploy the software. 

      firebase deploy

      Expected results and debugging

        • The dashboard should show the rxMqtt function in the log.
        • When the device publishes its state using MQTT, the function rxMqtt() gets called as shown in the log. 
          rxMqtt  Function execution took 363 ms, finished with status: 'ok'
rxMqtt  received topic
rxMqtt  Function execution started

      BigQuery to DataStudio

      Data Studio can visualize data through configurable charts and tables. We are going to use it to visualize the historic data stored in BigQuery. Alternatively, you can create a web app and pull the data using an HTTPS endpoint as outlined in Appendix A.

      Block diagram: Cloud BigQuery to DataStudio
      Block diagram: Cloud BigQuery to Data Studio

      Configure the dashboard

        • Add the data source history to report
        • Insert a time series (draw it at the top quarter of the canvas)
          • The time dimension will be pre-filled with timestamp
          • Metric: temp (and remove others); click on the "123" in front of the metric field, and select average
          • Data range: custom, include today and last 7 days
        • Copy and paste the temperature graph below it, and change the field to humidity.
        • Insert a table (draw it at the bottom half of the canvas)
          • Dimension: deviceId and timestamp (and remove others)
          • Metric: temp and humidity (and remove others); click on the "123" labels in front of the metric fields, and select average
          • Data range: custom, include today and last 7 days
        • To change the theme to dark
          • Edit > Select none
          • Theme > current Theme = Simple Dark

      Expected results

      This was one of the easiest visualization methods that I have encountered. Hopefully, you get a similar dashboard.

      Screen shot of visualization in Data Studio

      Google Assistent

      The players

      Dialogflow

      Dialogflow provides a server-side infrastructure to create conversational scenarios, then build advanced dialogues to manage the conversation flow with the user.

      Google Assistant

      Google Assistant combines machine learning, speech recognition and natural language understanding. It allows you to have a conversation with the device.

      Actions on Google

        • Import your project (in my case: esp-nodejs-mqtt)
        • ...
        • enable " Google HomeGraph"

      Dialog Flow

      We use Dialog Flow to to connect to Google Assistant. It will be able to recall the previous device readings and pass a instruction to the device. To get started, head over Cloud Console and create a project and prepare it for Message Queuing Telemetry Transport (MQTT). MQTT is a lightweight machine-to-machine publish/subscribe protocol, often used where a small code footprint is required and/or network bandwidth is at a premium.

      APPENDIX A: Cloud Storage to WebApp

      This near vanilla copy of Alvaro Viebrantz' WebApp code is provided for completeness [3]. The public web app uses Web Material Component and Chart.JS for the charts.

      Block diagram: Cloud BigQuery to WebApp
      Block diagram: Cloud BigQuery to WebApp

      Program the functionality

      Configure and setup a report generator that listens on HTTPS.

      1. Point Firebase to the BigQuery table 
        firebase functions:config:set ^
bigquery.datasetname="esp_nodejs_mqtt" ^
bigquery.tablename="history"
      2. Add code to the end of functions\index.js (in your firebase directory). This will
        • getWeek() functions as an HTTPS endpoint for the web app by supplying the last 7 days of data from BigQuery.
        // BigQuery and current state to HTTPS endpoint

const cors = require('cors')({ origin: true });

exports.getWeek = functions.https.onRequest( (req, res) => {
    const project = functions.config().firebase.projectId;
    const dataset = functions.config().bigquery.datasetname;
    const tablename = functions.config().bigquery.tablename;
    const table = '`' + project + '.' + dataset + '.' + tablename + '`';
    const query = `
    SELECT 
        TIMESTAMP_TRUNC(data.timestamp, HOUR, 'America/Los_Angeles') AS data_hour,
        avg(data.temp) as avg_temp,
        avg(data.humidity) as avg_hum,
        min(data.temp) as min_temp,
        max(data.temp) as max_temp,
        min(data.humidity) as min_hum,
        max(data.humidity) as max_hum,
        count(*) as data_points      
    FROM ${table} data
    WHERE data.timestamp > TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY)
    GROUP by data_hour
    ORDER by data_hour
    `;
    bigquery
    .query({
        query: query,
        useLegacySql: false
    })
    .then(results => {
        const rows = results[0];
        return cors(req, res, () => {
            res.json(rows);
        });
    })
    .catch(err => {
        res.status(500).send(err);
        console.error('ERROR:', err);
    });
});
      3. Install the modules referenced by functions\package.json. 
        cd functions
npm install
cd ..
      4. Update database.rules.json to allow public read access to the database. No ideal, as an alternative you can have the user authenticate using OAuth. 
        {
    "rules": {
    ".read": "true",
    ".write": "auth != null"
    }
}

      Deploy

      Time to see it all in action. 

      firebase deploy

      Expected results and debugging

        • The dashboard should show the getWeek function as evident from the log.
      2. When you access the HTTPS endpoint URL listed on the Dashboard, it should return something like shown below 
        [
    {"data_hour":{"value":"2018-03-11 03:00:00.000"},
    "avg_temp":23,"avg_hum":53.25,
    "min_temp":23,"max_temp":23,"min_hum":47,"max_hum":56,"data_points":4},
    {"data_hour":{"value":"2018-03-11 04:00:00.000"},
    "avg_temp":23,"avg_hum":54.2,
    "min_temp":23,"max_temp":23,"min_hum":53,"max_hum":55,"data_points":5},
    {"data_hour":{"value":"2018-03-11 05:00:00.000"},
    "avg_temp":23,"avg_hum":52.2,
    "min_temp":23,"max_temp":23,"min_hum":47,"max_hum":54,"data_points":5},
    {"data_hour":{"value":"2018-03-11 06:00:00.000"},
    "avg_temp":22.333333333333332,"avg_hum":54.666666666666664,
    "min_temp":22,"max_temp":23,"min_hum":54,"max_hum":55,"data_points":3}
]
        • Every time you access your the HTTPS URL listed on the Dashboard, getWeek() gets called. 
          getWeek Function execution took 1740 ms, finished with status code: 200
getWeek Function execution started

      WebApp

      (http://material-components-web.appspot.com/)

      1. Modify the public web page public\index.html 
        <!DOCTYPE html>
    <html>
        <head>
        <meta charset="utf-8">
        <meta name="viewport" content="width=device-width, initial-scale=1">
        <title>Welcome esp-nodejs-mqtt</title>

        <!-- update the version number as needed -->
        <script defer src="https://coertvonk.com/__/firebase/4.11.0/firebase-app.js"></script>
        <!-- include only the Firebase features as you need -->
        <script defer src="https://coertvonk.com/__/firebase/4.11.0/firebase-auth.js"></script>
        <script defer src="https://coertvonk.com/__/firebase/4.11.0/firebase-database.js"></script>
        <script defer src="https://coertvonk.com/__/firebase/4.11.0/firebase-messaging.js"></script>
        <script defer src="https://coertvonk.com/__/firebase/4.11.0/firebase-storage.js"></script>
        <!-- initialize the SDK after all desired features are loaded -->
        <script defer src="https://coertvonk.com/__/firebase/init.js"></script>
        <script defer src="https://cdn.firebase.com/libs/firebaseui/2.5.1/firebaseui.js"></script>
        
        <link type="text/css" rel="stylesheet" href="https://cdn.firebase.com/libs/firebaseui/2.5.1/firebaseui.css" />
        <script defer src="app.js"></script>    
        <link rel="stylesheet" href="https://fonts.googleapis.com/icon?family=Material+Icons">
        <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto+Mono">
        <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,400,500">
        <link rel="stylesheet" href="https://unpkg.com/material-components-web@latest/dist/material-components-web.min.css">
        <style>
            :root {
            --mdc-theme-primary: #0e4ead;
            --mdc-theme-background: #E0E0E0;
            } 
            main {
            margin: 0px auto;
            }
            .mdc-card {
            margin-bottom: 20px;
            padding: 8px;
            }
        </style>
        
        <style media="screen">
            body { background: #ECEFF1; color: rgba(0,0,0,0.87); font-family: Roboto, Helvetica, Arial, sans-serif; margin: 0; padding: 0; }
            #message { background: white; max-width: 360px; margin: 100px auto 16px; padding: 32px 24px; border-radius: 3px; }
            #message h2 { color: #ffa100; font-weight: bold; font-size: 16px; margin: 0 0 8px; }
            #message h1 { font-size: 22px; font-weight: 300; color: rgba(0,0,0,0.6); margin: 0 0 16px;}
            #message p { line-height: 140%; margin: 16px 0 24px; font-size: 14px; }
            #message a { display: block; text-align: center; background: #039be5; text-transform: uppercase; text-decoration: none; color: white; padding: 16px; border-radius: 4px; }
            #message, #message a { box-shadow: 0 1px 3px rgba(0,0,0,0.12), 0 1px 2px rgba(0,0,0,0.24); }
            #load { color: rgba(0,0,0,0.4); text-align: center; font-size: 13px; }
            @media (max-width: 600px) {
            body, #message { margin-top: 0; background: white; box-shadow: none; }
            body { border-top: 16px solid #ffa100; }
            }
        </style>
        </head>
        <body>
        <header class="mdc-toolbar mdc-toolbar--fixed">
            <div class="mdc-toolbar__row">
            <section class="mdc-toolbar__section mdc-toolbar__section--align-start">
                <span class="mdc-toolbar__title">esp-nodejs-mqtt - Cloud IoT Core</span>
            </section>
            <!--
                <section class="mdc-toolbar__section mdc-toolbar__section--align-end" role="toolbar">
                    <a href="#" class="material-icons mdc-toolbar__icon" aria-label="Download" alt="Download">more_vert</a>
                </section>
                -->
            </div>
        </header>
        <main class="mdc-toolbar-fixed-adjust">
            <div class="mdc-layout-grid">
            <div class="mdc-layout-grid__inner">
                <div class="mdc-layout-grid__cell mdc-layout-grid__cell--span-12">
                <div class="mdc-card">
                    <section class="mdc-card__primary">
                    <p id="load">Firebase SDK Loading…</p>
                    <h1 class="mdc-card__title mdc-card__title--large">Current data collected</h1>
                    </section>
                    <ul id="devices" class="mdc-list mdc-list--two-line mdc-list--avatar-list two-line-avatar-text-icon-demo">
                    <div role="progressbar" class="mdc-linear-progress mdc-linear-progress--indeterminate">
                        <div class="mdc-linear-progress__buffering-dots"></div>
                        <div class="mdc-linear-progress__buffer"></div>
                        <div class="mdc-linear-progress__bar mdc-linear-progress__primary-bar">
                        <span class="mdc-linear-progress__bar-inner"></span>
                        </div>
                        <div class="mdc-linear-progress__bar mdc-linear-progress__secondary-bar">
                        <span class="mdc-linear-progress__bar-inner"></span>
                        </div>
                    </div>
                    </ul>
                </div>
                </div>
                <div class="mdc-layout-grid__cell mdc-layout-grid__cell--span-6">
                <div class="mdc-card">
                    <section class="mdc-card__primary">
                    <h1 class="mdc-card__title mdc-card__title--large">Temperature - Last 7 days</h1>
                    </section>
                    <canvas id="tempLineChart"></canvas>
                    <div role="progressbar" id="tempLineChart_progress" class="mdc-linear-progress mdc-linear-progress--indeterminate">
                    <div class="mdc-linear-progress__buffering-dots"></div>
                    <div class="mdc-linear-progress__buffer"></div>
                    <div class="mdc-linear-progress__bar mdc-linear-progress__primary-bar">
                        <span class="mdc-linear-progress__bar-inner"></span>
                    </div>
                    <div class="mdc-linear-progress__bar mdc-linear-progress__secondary-bar">
                        <span class="mdc-linear-progress__bar-inner"></span>
                    </div>
                    </div>
                </div>
                </div>
                <div class="mdc-layout-grid__cell mdc-layout-grid__cell--span-6">
                <div class="mdc-card">
                    <section class="mdc-card__primary">
                    <h1 class="mdc-card__title mdc-card__title--large">Humidity - Last 7 days</h1>
                    </section>
                    <canvas id="humLineChart"></canvas>
                    <div role="progressbar" id="humLineChart_progress" class="mdc-linear-progress mdc-linear-progress--indeterminate">
                    <div class="mdc-linear-progress__buffering-dots"></div>
                    <div class="mdc-linear-progress__buffer"></div>
                    <div class="mdc-linear-progress__bar mdc-linear-progress__primary-bar">
                        <span class="mdc-linear-progress__bar-inner"></span>
                    </div>
                    <div class="mdc-linear-progress__bar mdc-linear-progress__secondary-bar">
                        <span class="mdc-linear-progress__bar-inner"></span>
                    </div>
                    </div>
                </div>
                </div>
            </div>
            </div>
        </main>
        <script src="https://unpkg.com/material-components-web@latest/dist/material-components-web.min.js"></script>
        <script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/2.5.0/Chart.bundle.min.js"></script>
        <script>
            window.mdc.autoInit();
        </script>
        </body>
    </html>
      2. Add JavaScript code for the web page to public\app.js 
        if( document.readyState === 'complete' ) {
    myInitCode();
} else {
    document.addEventListener('DOMContentLoaded', function () {
        myInitCode();
    });
}

function myInitCode() {
    // attach an async callback to read the data    
    firebase.auth().onAuthStateChanged(user => {
    if (user) {
            console.log("AuthStateChanged - user signed");
    } else {
            console.log("AuthStateChanged - no user signed in");
    }
    });
    
    firebase.database().ref('devices').on('value', snapshot => {
    console.log("value changed");
    console.log(snapshot.val());
    
    let devices = snapshot.val();
    console.log(devices);
    let devicesEl = document.getElementById('devices');
    devicesEl.innerHTML = '';
    
    for (var key in devices) {
            let deviceState = devices[key];
            let li = document.createElement('li');
            li.className = 'mdc-list-item';
            li.innerHTML = `
        <span class="mdc-list-item__start-detail grey-bg" role="presentation">
            <i class="material-icons" aria-hidden="true">cloud</i>
        </span>
        <span class="mdc-list-item__text">
            Station #${key}
            <span class="mdc-list-item__text__secondary">
                ${deviceState.temp} C°/${deviceState.humidity} %
            </span>
            <span class="mdc-list-item__text__secondary">
                Last updated: ${new Date(deviceState.timestamp).toLocaleString()}
            </span>
        </span>
        `;
        
            devicesEl.appendChild(li);
    }
    }, function(errorObject) {
    console.log("The read failed: " + errorObject.code);
    });
    
    fetchReportData();
}

const reportDataUrl = 'https://us-central1-YOURPROJECTID.cloudfunctions.net/getWeek';

function fetchReportData() {
    try {
    fetch(reportDataUrl)
            .then(res => res.json())
            .then(rows => {
        var maxTempData = rows.map(row => row.max_temp);
        var avgTempData = rows.map(row => row.avg_temp);
        var minTempData = rows.map(row => row.min_temp);
        
        var maxHumData = rows.map(row => row.max_hum);
        var avgHumData = rows.map(row => row.avg_hum);
        var minHumData = rows.map(row => row.min_hum);
        
        var labels = rows.map(row => row.data_hour.value);
        
        buildLineChart(
            'tempLineChart',
            'Temperature in C°',
            labels,
            '#E64D3D',
            avgTempData
        );
        buildLineChart(
            'humLineChart',
            'Humidity in %',
            labels,
            '#0393FA',
            avgHumData
        );
            });
    } catch (e) {
    alert('Error getting report data');
    }
}

function buildLineChart(el, label, labels, color, avgData) {
    const elNode = document.getElementById(el);
    new Chart(elNode, {
    type: 'line',
    data: {
            labels: labels,
            datasets: [
        {
            label: label,
            data: avgData,
            borderWidth: 1,
            fill: true,
            spanGaps: true,
            lineTension: 0.2,
            backgroundColor: color,
            borderColor: '#3A4250',
            pointRadius: 2
        }
            ]
    },
    options: {
            responsive: true,
            scales: {
        xAxes: [
            {
            type: 'time',
            distribution: 'series',
            ticks: {
                source: 'labels'
            }
            }
        ],
        yAxes: [
            {
            scaleLabel: {
                display: true,
                labelString: label
            },
            ticks: {
                stepSize: 0.5
            }
            }
        ]
            }
    }
    });
    
    const progressEl = document.getElementById(el + '_progress');
    progressEl.remove();
    
    try {
        let app = firebase.app();
        let features = ['auth', 'database', 'messaging', 'storage'].filter(feature => typeof app[feature] === 'function');
        document.getElementById('load').innerHTML = `Firebase SDK loaded with ${features.join(', ')}`;
    } catch (e) {
        console.error(e);
        document.getElementById('load').innerHTML = 'Error loading the Firebase SDK, check the console.';
    }
    
}

      Deploy 

      firebase deploy

      Expected results and debugging

      The WebApp should look similar as shown in the figure below

      WebApp on a Nexus 5X

      Credits

      [1] Talk to your CD player using Google Home
      Sander Vonk, 2017-Oct-15, retrieved 2018-02-16.
      coertvonk.com/sw/embedded/google-home-ifttt-esp8266-integration-23066
      [2] Itead Sonoff S20 + MQTT + Google Assistant
      Coert Vonk, 2018-Mar-01, retrieved 2018-Mar-01.
      github.com/cvonk/esp8285-mqtt_light
      [3] Build a Weather Station using Google Cloud IoT Core and MongooseOS
      Alvaro Viebrantz, 2017-Oct-15, retrieved 2018-Feb-16.
      medium.com/google-cloud/build-a-weather-station-using-google-cloud-iot-core-and-mongooseos-7a78b69822c5
      [4] Overview of Internet of Things
      Google, 2017-Sep-28, retrieved 2018-Mar-17.
      cloud.google.com/solutions/iot-overview

      create and register device by hand

      Ameba : Getting Started With Google Cloud IoT

      Google Assistant integration: https://codelabs.developers.google.com/codelabs/assistant-codelab/index.html?index=..%2F..%2Findex#0

Copyright © 1996-2022 Coert Vonk, All Rights Reserved