RS-485 bus

The pool controller uses a “RS-485” bus to communicate with devices such as the pump and chlorinator. This is an industry standard two-wire bus with differential signaling. Devices are daisy-chained with 120 Ω termination resistors on both ends to limit reflections.

By snooping the traffic on this bus, we can learn about the status of the controller. We can also observe commands that the controller sends to the devices and the subsequent responses that it receives.

Opening the pool controller with a Phillips-head screwdriver

Inside the controller

The easiest place to access the bus is from the inside of the pool controller.

THIS PROJECT IS OFFERED AS IS. IF YOU USE IT YOU ASSUME ALL RISKS. NO WARRENTIES. PROCEED AT YOUR OWN RISK! Always make sure to at the very least turn off the power while working on your pool equipment. Work carefully, THERE IS ALWAYS A RISK OF BREAKING YOUR POOL EQUIPMENT.

That being said, the RS-485 header can be found on the back of the control board. There are most likely wires already connected that go to devices such as the pump and chlorinator.

Inside of controller
Inside of controller with the connector removed

To minimize electromagnetic interference, use two twisted pairs from something like a CAT-5e cable to connect the A/B pair and the +15V/GND power pair to the interface as shown in the table below. To minimize the risk of shorting the power, you probably don’t want to connect the power signal just yet.

Controller RS-485 Cat5
Com Port Name Name Idle state Wire Pair
Green -DATA A negative pair 1
Yellow +DATA B positive pair 1
Red +15 V n/a pair 2
Black GND n/a pair 2

RS-485 connection

With our cable connected, we can examine the signals. The shortest pulse length is about 104 µsec. From this, the data rate follows as 1/0.000104 = 9600 baud. The exact flavor of the data bus is shown below.

Name Value
baud rate 9600
data bits 8
parity none
stop bits 1

Such a low data rate implies that we don’t have to be very concerned about daisy-chaining the devices. In other words, it is OK to just wiretap the signal on the controller.

1 bit takes ≈104 µsec
RS-485 transceiver breakout

When we connect the differential signal to a RS-485 transceiver such as the MAX485, we get regular TTL levels.

Single message
Messages repeat every 2 sec

TTL to CMOS signals

The microcontrollers that we consider use CMOS voltages (3.3 V). For the prototypes, we used an existing breakout board, but replaced the MAX485 with a MAX3485 chip. While we’re at it, we’ll also remove the 10 kΩ CMOS pullup resistors (R1R4), but leave the RS-485 bias and bus termination resistors.

Breakout schematic
Source: yourduino.com
RS485 with new 3.3V part

Knowing about the interface and baud rate, we can move on to building a system that interprets the proprietary message protocol. Continue reading to learn about the hardware design.

Google IoT meets ESP32, MQTT, Node JS

ESP32, MQTT, NodeJS and Google IoT

This tutorial gives you a glimpse on integrating IoT devices with Google’s ecosystem. This approach is scalable to a great many devices, has security designed in an only relies on the IoT device and Google Cloud Platform.

This approach is more involved compared to our earlier approaches [1,2] that relied on many in between systems. This time around, the only programming will be in JavaScript (Node JS). No opening ports on the firewall, no IFT3. Plain and simple, ‘though never as simple as getting as opening the door to see how the weather is.

This guide was inspired by Alvaro Viebrantz’ post [3]. With MongooseOS and Google Cloud IoT still in beta, it occasionally felt like the ground was moving under my feet, but due patience and persistence makes engineering great. With this write-up, I try to make it less daunting while providing an understanding of the inner workings.

This write-up is structured as a step-by-step guide to help you understand the elements. If you prefer to find your own way around, feel free to head over to the GitHub repository (no longer life!). We use Windows 10, but the instructions apply to other operating systems as well. Just note that in Windows PowerShell is the character ` is used to split up long lines.

Design

Goal

The design needs to be scalable, secure and fit for low-power SoC applications.

The lineup

IoT integration with Google Cloud

Message Protocol

The key ingredient is the MQTT protocol. The abbreviation MQTT stands for Message Queue Telemetry Transport, where the keyword is telemetry, or remote monitoring. Its publish/subscription model makes it an efficient protocol that is well suited for low power devices. By creating topics, you can enable different parts of your application to subscribe to specific streams of data. MQTT runs on the TCP transport that provides reliable delivery, but also requires devices to be awake at least some percentage of the time. The MQTT protocol is an efficient binary industry standard that allows constrained devices to send real-time telemetry as well as immediately receive messages sent from the cloud. I designed wearable products that used a phone to bridge from BTLE to MQTT in large scale deployment on AWS IoT. This time around, I will keep things as simple as possible while still keeping the design criteria front and center.

The “thing”

To keep things simple we an Espressif ESP32 System-on-a-Chip (SoC) that has builtin WiFi support. It would not be my first choice for low power, but it is cheap and is well supported. We connect a simple temperature and humidity sensor to generate some data. Feel free to use whatever other sensor comes to mind.

Google Cloud Platform

On the backend, we use the Google Cloud Platform that recently introduced MQTT support as “Cloud IoT”. The SoC will publish its readings to Cloud IoT, from where they are stored. For visualization we use the Data Studio tool. The block diagram at the beginning of this chapter shows the data flow. [4]

Cloud IoT Core

IoT Core handles registration, authentication and authorization inside the Cloud Platform resource hierarchy. It connects to Cloud Pub/Sub to provide a secure MQTT broker service. It also provides the ability to send device configuration to devices.

Cloud Pub/Sub

Google Cloud Pub/Sub implements MQTT by providing a globally durable message ingestion service. Cloud Pub/Sub can act like a shock absorber and rate leveler for both incoming data streams and application architecture changes. Many devices have limited ability to store and retry sending telemetry data. Cloud Pub/Sub scales to handle data spikes that can occur when swarms of devices respond to events in the physical world, and buffers these spikes to help isolate them from applications monitoring the data. [src]

Firebase real-time database

Firebase Realtime Database is a cloud-hosted NoSQL database that stores data as JSON. We use it to store the last value read from the sensors.

Cloud BigQuery

Cloud BigQuery enables standard SQL queries for interactive analysis of massively large datasets working in conjunction with Google Storage.

Firebase Functions

Firebase Functions lets you run backend code that responds to events in the Cloud. We will have it responds to data published to a Pub/Sub topic, or a request from DialogFlow to a HTTPS endpoint.

Data Studio

Data Studio can visualize data through configurable charts and tables. We are going to use it to visualize the historic data stored in BigQuery.

Device to Cloud Pub/Sub

Block diagram: Device to Cloud Pub/Sub

Google Cloud

We use Google Cloud to store and access the readings from our IoT device. To get started, head over Cloud Console and create a project and prepare it for Message Queuing Telemetry Transport (MQTT). MQTT is a lightweight machine-to-machine publish/subscribe protocol, often used where a small code footprint is required and/or network bandwidth is at a premium.

    • + to create a project with a unique name (this post assumes the name “esp-nodejs-mqtt”)
    • Enable billing. Google Cloud has a generous free quota, but read the pricing and terms to ensure you are comfortable with them.
    • enable “ Google Cloud IoT”
    • enable “ Google Cloud Pub/Sub”
  1. Connect your services with reliable, many-to-many, asynchronous messaging hosted on Google’s infrastructure.
    • Name: mqtt-topic
  2. A device registry allows you to group and set properties that they all share, such as connection protocol, data storage location, and CLoud Pub/Sub topics.
    • Registry ID: mqtt-registry
    • Cloud region: us-central1 (whatever is closer to you)
    • Protocol: MQTT, HTTP
    • Default telemetry topic: mqtt-topic

We will add the device and its public key in the next section.

The “thing”

Our “thing” will be an Espressif ESP32 board with an integrated USB-serial, such as the Adafruit HUZZAH32. No particular reason, besides that it is cheap and has build-in WiFi. A temperature and humidity sensor is connected to GPIO21. If you use an ESP8266, connect the sensor to GPIO4.

On the ESP32, we use a off-the-shelf firmware that can be programmed using JavaScript. The firmware, called Mongoose OS, comes with a library to interface with Google IoT Core. [github, github].

Tools

The command line tool mos generates custom firmware and uploads it to the ESP32. The tool also gives easy access to the serial debug port and SPIFFS file system. For integration with Cloud IoT, it relies on Google’s gcloud command line tool.

gcloud

We will start with installing the Google Cloud tool and linking it to our project.

  1. Open a command terminal, create a project directory and make that the active directory.
  2. Install the gcloud command line tool as described in Cloud SDK.
    • Choose to include all components (incl. Beta components).
    • Accept all post-install options. Once it executes gcloud init, choose to
      • login (to Google), and follow the directions,
      • pick your cloud project (for us that was “esp-nodejs-mqtt”).
      • It should finish with “Your Google Cloud SDK is configured and ready to use”
  3. Give Cloud IoT access your project
    gcloud projects add-iam-policy-binding esp-nodejs-mqtt `
            --member=serviceAccount:cloud-iot@system.gserviceaccount.com `
            --role=roles/pubsub.publisher
    Remember to use your own project id (see cloud projects list).

mos

Install the Mongoose OS tool and install the Cloud IoT’s CA certificate

  1. Open a command terminal, and make your project directory the current directory.
  2. Start with some boilerplate code (install git if you haven’t done so already)
    git clone https://github.com/mongoose-os-apps/empty firmware
  3. Install the mos tool in the firmware directory according to its instructions.
  4. Downgrade to version 1.22 because this seems to be the last stable version
    firmware\mos update 1.22
  5. Connect the ESP32 to your computer using USB.
  6. Make the firmware you current directory.
  7. Retrieve Google Cloud’s CA certificate using a *nix system with openssl installed
    echo -n | openssl s_client -showcerts -connect mqtt.googleapis.com:8883 | \
            sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > ca.pem
    Copy this ca.pem to firmware\fs\ca.pem

Program the device

With Cloud IoT setup, it is time to tell the device what to do. For this tutorial, the device will measure the temperature and humidity every 5 seconds. Every 15 minutes it will publish the readings to Cloud IoT as topic /devices/DEVICENAME/events/.

  1. Update the build description firmware\mos.yml with the apps’ configuration and libraries
    author: Alvaro Viebrantz, modified by Coert Vonk
    description: esp-nodejs-iot firmware
    version: 1.0
    
    libs_version: ${mos.version}
    modules_version: ${mos.version}
    mongoose_os_version: ${mos.version}
    
    # List of files/directories with C sources. No slashes at the end of dir names
    sources:
        - src
    
    # Files from these dirs will be copied to the device filesystem
    filesystem:
        - fs
    
    # Custom configuration entries, settable via "device configuration"
    config_schema:
        - ["i2c.enable", true]
        - ["app", "o", { title: "App custom settings"}]
        - ["app.dht", "i", 21, { title: "DHT pin"}]
    
    # List of libraries used by this app, in order of initialisation
    libs:
    # use fs\ca.pem instead of https://github.com/mongoose-os-libs/ca-bundle
    # - origin: https://github.com/mongoose-os-libs/ca-bundle
        - origin: https://github.com/mongoose-os-libs/rpc-service-config
        - origin: https://github.com/mongoose-os-libs/rpc-service-fs
        - origin: https://github.com/mongoose-os-libs/rpc-uart
        - origin: https://github.com/mongoose-os-libs/wifi
    # libs for the current app
        - origin: https://github.com/mongoose-os-libs/mqtt
        - origin: https://github.com/mongoose-os-libs/i2c
        - origin: https://github.com/mongoose-os-libs/gcp
        - origin: https://github.com/mongoose-os-libs/mjs
        - origin: https://github.com/mongoose-os-libs/dht
    
    # Used by the mos tool to catch mos binaries incompatible with this file format
    manifest_version: 2017-05-18
  2. When using the ESP8266, add a device-type specific build description firmware\mos_esp8266.yml to change the DHT pin to GPIO4
    # ESP8266 specific configuration entries
    config_schema:
        - ["app.dht", "i", 4, { title: "DHT pin"}]
        
  3. Create a JavaScript program firmware\fs\init.js to read the sensor and periodically publish it to Cloud IoT. The program assigns the sensor pin based on the app.dht property in the yml build description.
    load('api_config.js');
    load('api_mqtt.js');
    load('api_timer.js');
    load('api_dht.js');
    load('api_config.js');
    load('api_gpio.js');
    load('api_mqtt.js');
    load('api_net.js');
    load('api_sys.js');
    
    let appName = 'init.js             ';
    let deviceName = Cfg.get('device.id');
    
    // The MQTT topic name is required to be in the format below. The topic name must end
    // in 'state' to publish state and 'events' to publish telemetry. Note that this is
    // not the same as the device registry's Cloud Pub/Sub topic.
    let topic = '/devices/' + deviceName + '/events';
    print(appName, 'Topic:', topic);
    
    let isConnected = false;
    let dhtPin = Cfg.get('app.dht');
    let dht = DHT.create(dhtPin, DHT.DHT11);
    
    function getInfo() {
        return JSON.stringify({
            total_ram: Sys.total_ram() / 1024,
            free_ram: Sys.free_ram() / 1024,
            temp: dht.getTemp(),
            hum: dht.getHumidity()
        });
    };
    
    function publishData() {
        if (MQTT.pub(topic, getInfo())) {
            print(appName, 'Published');
        }
    }
    
    Timer.set(
        15 * 60 * 1000, true,
        function() {
            if (isConnected) {
                print(appName, '5 minute timer, CONNECTED > PUBLISH');
                publishData();
            }
        }, null
    );
    
    Timer.set(
        5 * 1000, true,
        function() {
            print(appName, 'Info:', getInfo());
        }, null
    );
    
    MQTT.setEventHandler(function(conn, ev, data) {
        if (ev !== 0) {
            let evs = '?';
            if (ev === MQTT.EV_CONNACK) {
                evs = 'CONNACK';   // Connection to broker has been established
            } else if (ev === MQTT.EV_PUBLISH) {
                evs = 'PUBLISH';   // msg published to topics we are subscribed to
            } else if (ev === MQTT.EV_PUBACK) {
                evs = 'PUBACK';    // ACK for publishing of a message with QoS>0
            } else if (ev === MQTT.EV_SUBACK) {
                evs = 'SUBACK';    // ACK for a subscribe request
            } else if (ev === MQTT.EV_UNSUBACK) {
                evs = 'UNSUBACK';  // ACK for an unsubscribe request
            } else if (ev === MQTT.EV_CLOSE) {
                evs = 'CLOSE';     // connection to broker was closed
            }
            print(appName, 'MQTT event:', evs, ev);
            if (ev === MQTT.EV_CONNACK) {
                print(appName, 'MQTT CONNACK > PUBLISHING');
                isConnected = true;
                publishData();
            }
        }
    }, null);
  4. Create a binary and file system for the device
    .\mos build --arch esp32
    .\mos flash

Deploy

Register with Cloud IoT and configure WiFi

  1. Register the device with Cloud IoT. This single command will
    • use the device id to generate a key pair,
    • upload the private key to the device,
    • add gcp and mqtt configuration to the conf9.json file on the device,
    • (re)create the device with its public key to Cloud IoT.
    .\mos gcp-iot-setup --gcp-project esp-nodejs-mqtt --gcp-region us-central1 ^
            --gcp-registry mqtt-registry
    Once more, remember to use your own project id and geographic region. If successful, it will show the JSON gcp and mqtt configuration. If the command hangs, make sure that GPIO12 is not pulled down.
  2. Add the WiFi credentials to the configuration conf9.json file on the device
    .\mos wifi "yourWiFiSSID" "yourWiFiPasswd"

Expected results and debugging

The device should be connecting to Cloud IoT as evident from the device’s debug trace and current value pulled from Cloud IoT

  1. This should show when the last telemetry event was received, and the device’s ES256 public key.
  2. Show the debug output of the device
    .\mos console
    • If things go well you will see debug messages from the device such as

      esp32_mgos_init      firmware 1.0 (20180309-003629/???)
      mgos_i2c_create      I2C GPIO init ok (SDA: 32, SCL: 33)
      init.js              Topic:  /devices/esp32_F00F00/events 
      init.js              Net event: CONNECTING 1 
      init.js              Net event: CONNECTED 2 
      init.js              Net event: GOT_IP 3 
      mgos_mqtt_global_con MQTT connecting to mqtt.googleapis.com:8883
      mgos_sntp_query      SNTP query to time.google.com
      mgos_mqtt_ev         MQTT TCP connect ok (0)
      mgos_sntp_ev         SNTP reply from 216.239.35.4: time 1520555855.896951, loc...
      init.js              Info: {"hum":55,"temp":21,"free_ram":201.261719,"total_ra... 
      mgos_mqtt_ev         MQTT CONNACK 0
      init.js              MQTT event: CONNACK 202 
      init.js              MQTT CONNACK > PUBLISHING 
      init.js              Published 
      init.js              Info: {"hum":48,"temp":21,"free_ram":202.527344,"total_ra... 
      init.js              Info: {"hum":57,"temp":21,"free_ram":202.527344,"total_ra,..
      Debug messages from the ESP32 device

      If things don’t go so smooth, you’re best starting point is using the MongooseOS web interface (.\mos ui). A common cause of errors is uploading the firmware after configuring the device causing the configuration to be overwritten. Net connection errors may be the effect of missing or incorrect WiFi credentials in conf9.json on the device. If you encounter MQTT CONNACK 4 messages, check the ca.pem certificate. This should match the certificate pulled using openssl earlier. Also try switching to version 1.22 of the mos tool if aren’t using this already. CONNACK 5 on the other hand, hints at using the wrong registry or project name.

      Cloud Pub/Sub to BigQuery

      We use Cloud BigQuery to store the device data. Google BigQuery is a web service that lets you do interactive analysis of massive datasets—up to billions of rows.

      Block diagram: Cloud Pub/Sub to BigQuery and Firebase Store

      BigQuery storage

      To get started, head over to BigQuery Console and create a dataset and table.

        • Dataset ID: esp_nodejs_mqtt (your project name)
        • Data location: US (whatever is closer)
        • Data expiration: Never
        • Source Data: Create empty table
        • Destination table name: esp_node_js.raw_data
        • Schema fields:
          deviceId  STRING     REQUIRED
              timestamp TIMESTAMP  REQUIRED
              temp      FLOAT      REQUIRED
              humidity  FLOAT      REQUIRED
        • Partitioning type = Day

      Firebase functions

      Firebase provides easy access to Google Cloud services intended for mobile or web.

      Link Firebase to your Cloud project using the Firebase Console

        • Project name: esp-node-js-mqtt (your project name from the pull down list)
        • Add Firebase Confirm plan

      node tool

      Start with installing Node JavaScript (NodeJS).

      1. Open a command terminal and make your project directory the current directory. Create a subdirectory (e.g. firebase and make that the current directory.
        • Firebase doesn’t tolerate spaces in the file path. On Windows, you can work around this creating a symbolic link (mklink /d) or map a virtual drive to a UNC path (net use).
      2. Install the LTS version of NodeJS from nodejs.org with the default options.
      3. Update the NodeJS package manager from a command terminal
        npm i npm

      firebase tool

      1. Install the NodeJS package “firebase”. We install the bleeding edge to work around issue #610. The -g parameter causes the modules to be installed globally (in %APPDATA%\npm).
        npm install -g git://github.com/firebase/firebase-tools#master
        The install may warn that node-uuid has been superseded with uuid.
      2. Sign in to your Google account using Firebase
        firebase login
      3. Initialize the project directory
        firebase init
        Make the following selections
        • Project setup
          • install all except FireStore,
          • select your project as the default Firebase project (in our case esp-nodejs-mqtt).
        • Database setup
          • database rules: database.rules.json.
        • Functions setup
          • language to use: JavaScript,
          • use ESLint: yes,
          • install dependencies with npm: yes.
          • This creates the files functions\package.json, functions\.eslintrc.json and functions\index.js.
        • Hosting setup
          • public directory: public,
          • rewrite all urls to /index.html: no.
          • This creates the files public/404.html and public/index.html.
        • Storage setup
          • file for storage rules: storage.rules.
        • It finishes with creating the configuration file firebase.json and project info .firebaserc.

      Program the functionality

      Time to write some software! Configure and setup a listener in Firebase to store the MQTT messages in BigQuery, and maintain the device’s state in Firebase realtime store. This gives the cloud instant access event when the device is in low-power sleep mode and/or disconnected.

      1. Point Firebase to the BigQuery table
        firebase functions:config:set ^
        bigquery.datasetname="esp_nodejs_mqtt" ^
        bigquery.tablename="history"
      2. Update the file functions\index.js (in your firebase directory). This will
        • rxMqtt() receives data from Cloud Pub/Sub, forward it to Cloud BigQuery and stores the last value in Firebase realtime storage.
        const functions = require('firebase-functions');
        const admin = require('firebase-admin');
        admin.initializeApp(functions.config().firebase);
        
        const bigquery = require('@google-cloud/bigquery')();
        const db = admin.database();
        const cors = require('cors')({ origin: true });
        
        // MQTT to BigQuery and maintain current state in Firebase
        
        function updateCurrentStatusInFirebase(data) {
            return db.ref(`/devices/${data.deviceId}`).set({
                humidity: data.humidity,
                temp: data.temp,
                timestamp: data.timestamp
            });
        }
            
        function insertIntoBigQuery(data) {
            const dataset = bigquery.dataset(functions.config().bigquery.datasetname);
            const table = dataset.table(functions.config().bigquery.tablename);
            return table.insert(data);
        }
        
        exports.rxMqtt = functions.pubsub
            .topic('mqtt-topic')
            .onPublish(event => {
            console.log('received topic');
                const attributes = event.data.attributes;
                const message = event.data.json;
                const deviceId = attributes['deviceId'];
                const data = {
                    humidity: message.hum,
                    temp: message.temp,
                    deviceId: deviceId,
                    timestamp: event.timestamp
                };
                if ( message.hum < 0 || message.hum > 100 ||
                        message.temp > 100 || message.temp < -50 ) {
                    return null;  // ignore false readings
                }
                return Promise.all([
                    insertIntoBigQuery(data),
                    updateCurrentStatusInFirebase(data)
                ]);
            });[/code]
                
      3. Install the modules referenced by functions\package.json. cd functions npm install cd ..

      Deploy

      The moment of truth is here: time to deploy the software.

      firebase deploy

      Expected results and debugging

        • The dashboard should show the rxMqtt function in the log.
        • When the device publishes its state using MQTT, the function rxMqtt() gets called as shown in the log.
          rxMqtt  Function execution took 363 ms, finished with status: 'ok'
          rxMqtt  received topic
          rxMqtt  Function execution started

      BigQuery to DataStudio

      Data Studio can visualize data through configurable charts and tables. We are going to use it to visualize the historic data stored in BigQuery. Alternatively, you can create a web app and pull the data using an HTTPS endpoint as outlined in Appendix A.

      Block diagram: Cloud BigQuery to Data Studio

      Configure the dashboard

        • Add the data source history to report
        • Insert a time series (draw it at the top quarter of the canvas)
          • The time dimension will be pre-filled with timestamp
          • Metric: temp (and remove others); click on the "123" in front of the metric field, and select average
          • Data range: custom, include today and last 7 days
        • Copy and paste the temperature graph below it, and change the field to humidity.
        • Insert a table (draw it at the bottom half of the canvas)
          • Dimension: deviceId and timestamp (and remove others)
          • Metric: temp and humidity (and remove others); click on the "123" labels in front of the metric fields, and select average
          • Data range: custom, include today and last 7 days
        • To change the theme to dark
          • Edit > Select none
          • Theme > current Theme = Simple Dark

      Expected results

      This was one of the easiest visualization methods that I have encountered. Hopefully, you get a similar dashboard.

      Screen shot of visualization in Data Studio

      Google Assistent

      The players

      Dialogflow

      Dialogflow provides a server-side infrastructure to create conversational scenarios, then build advanced dialogues to manage the conversation flow with the user.

      Google Assistant

      Google Assistant combines machine learning, speech recognition and natural language understanding. It allows you to have a conversation with the device.

      Actions on Google

        • Import your project (in my case: esp-nodejs-mqtt)
        • ...
        • enable " Google HomeGraph"

      Dialog Flow

      We use Dialog Flow to to connect to Google Assistant. It will be able to recall the previous device readings and pass a instruction to the device. To get started, head over Cloud Console and create a project and prepare it for Message Queuing Telemetry Transport (MQTT). MQTT is a lightweight machine-to-machine publish/subscribe protocol, often used where a small code footprint is required and/or network bandwidth is at a premium.

      APPENDIX A: Cloud Storage to WebApp

      This near vanilla copy of Alvaro Viebrantz' WebApp code is provided for completeness [3]. The public web app uses Web Material Component and Chart.JS for the charts.

      Block diagram: Cloud BigQuery to WebApp

      Program the functionality

      Configure and setup a report generator that listens on HTTPS.

      1. Point Firebase to the BigQuery table
        firebase functions:config:set ^
        bigquery.datasetname="esp_nodejs_mqtt" ^
        bigquery.tablename="history"
      2. Add code to the end of functions\index.js (in your firebase directory). This will
        • getWeek() functions as an HTTPS endpoint for the web app by supplying the last 7 days of data from BigQuery.
        // BigQuery and current state to HTTPS endpoint
        
        const cors = require('cors')({ origin: true });
        
        exports.getWeek = functions.https.onRequest( (req, res) => {
            const project = functions.config().firebase.projectId;
            const dataset = functions.config().bigquery.datasetname;
            const tablename = functions.config().bigquery.tablename;
            const table = '`' + project + '.' + dataset + '.' + tablename + '`';
            const query = `
            SELECT 
                TIMESTAMP_TRUNC(data.timestamp, HOUR, 'America/Los_Angeles') AS data_hour,
                avg(data.temp) as avg_temp,
                avg(data.humidity) as avg_hum,
                min(data.temp) as min_temp,
                max(data.temp) as max_temp,
                min(data.humidity) as min_hum,
                max(data.humidity) as max_hum,
                count(*) as data_points      
            FROM ${table} data
            WHERE data.timestamp > TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY)
            GROUP by data_hour
            ORDER by data_hour
            `;
            bigquery
            .query({
                query: query,
                useLegacySql: false
            })
            .then(results => {
                const rows = results[0];
                return cors(req, res, () => {
                    res.json(rows);
                });
            })
            .catch(err => {
                res.status(500).send(err);
                console.error('ERROR:', err);
            });
        });
      3. Install the modules referenced by functions\package.json.
        cd functions
        npm install
        cd ..
      4. Update database.rules.json to allow public read access to the database. No ideal, as an alternative you can have the user authenticate using OAuth.
        {
            "rules": {
            ".read": "true",
            ".write": "auth != null"
            }
        }

      Deploy

      Time to see it all in action.

      firebase deploy

      Expected results and debugging

        • The dashboard should show the getWeek function as evident from the log.
      1. When you access the HTTPS endpoint URL listed on the Dashboard, it should return something like shown below
        [
            {"data_hour":{"value":"2018-03-11 03:00:00.000"},
            "avg_temp":23,"avg_hum":53.25,
            "min_temp":23,"max_temp":23,"min_hum":47,"max_hum":56,"data_points":4},
            {"data_hour":{"value":"2018-03-11 04:00:00.000"},
            "avg_temp":23,"avg_hum":54.2,
            "min_temp":23,"max_temp":23,"min_hum":53,"max_hum":55,"data_points":5},
            {"data_hour":{"value":"2018-03-11 05:00:00.000"},
            "avg_temp":23,"avg_hum":52.2,
            "min_temp":23,"max_temp":23,"min_hum":47,"max_hum":54,"data_points":5},
            {"data_hour":{"value":"2018-03-11 06:00:00.000"},
            "avg_temp":22.333333333333332,"avg_hum":54.666666666666664,
            "min_temp":22,"max_temp":23,"min_hum":54,"max_hum":55,"data_points":3}
        ]
        • Every time you access your the HTTPS URL listed on the Dashboard, getWeek() gets called.
          getWeek Function execution took 1740 ms, finished with status code: 200
          getWeek Function execution started

      WebApp

      (http://material-components-web.appspot.com/)

      1. Modify the public web page public\index.html
        
            
                
                
                
                Welcome esp-nodejs-mqtt
        
                
                
                
                
                
                
                
                
                
                
                
                
                    
                
                
                
                
                
                
                
                
                
                
        esp-nodejs-mqtt - Cloud IoT Core

        Firebase SDK Loading…

        Current data collected

        Temperature - Last 7 days

        Humidity - Last 7 days

      2. Add JavaScript code for the web page to public\app.js
        if( document.readyState === 'complete' ) {
            myInitCode();
        } else {
            document.addEventListener('DOMContentLoaded', function () {
                myInitCode();
            });
        }
        
        function myInitCode() {
            // attach an async callback to read the data    
            firebase.auth().onAuthStateChanged(user => {
            if (user) {
                    console.log("AuthStateChanged - user signed");
            } else {
                    console.log("AuthStateChanged - no user signed in");
            }
            });
            
            firebase.database().ref('devices').on('value', snapshot => {
            console.log("value changed");
            console.log(snapshot.val());
            
            let devices = snapshot.val();
            console.log(devices);
            let devicesEl = document.getElementById('devices');
            devicesEl.innerHTML = '';
            
            for (var key in devices) {
                    let deviceState = devices[key];
                    let li = document.createElement('li');
                    li.className = 'mdc-list-item';
                    li.innerHTML = `
                
                    
                
                
                    Station #${key}
                    
                        ${deviceState.temp} C°/${deviceState.humidity} %
                    
                    
                        Last updated: ${new Date(deviceState.timestamp).toLocaleString()}
                    
                
                `;
                
                    devicesEl.appendChild(li);
            }
            }, function(errorObject) {
            console.log("The read failed: " + errorObject.code);
            });
            
            fetchReportData();
        }
        
        const reportDataUrl = 'https://us-central1-YOURPROJECTID.cloudfunctions.net/getWeek';
        
        function fetchReportData() {
            try {
            fetch(reportDataUrl)
                    .then(res => res.json())
                    .then(rows => {
                var maxTempData = rows.map(row => row.max_temp);
                var avgTempData = rows.map(row => row.avg_temp);
                var minTempData = rows.map(row => row.min_temp);
                
                var maxHumData = rows.map(row => row.max_hum);
                var avgHumData = rows.map(row => row.avg_hum);
                var minHumData = rows.map(row => row.min_hum);
                
                var labels = rows.map(row => row.data_hour.value);
                
                buildLineChart(
                    'tempLineChart',
                    'Temperature in C°',
                    labels,
                    '#E64D3D',
                    avgTempData
                );
                buildLineChart(
                    'humLineChart',
                    'Humidity in %',
                    labels,
                    '#0393FA',
                    avgHumData
                );
                    });
            } catch (e) {
            alert('Error getting report data');
            }
        }
        
        function buildLineChart(el, label, labels, color, avgData) {
            const elNode = document.getElementById(el);
            new Chart(elNode, {
            type: 'line',
            data: {
                    labels: labels,
                    datasets: [
                {
                    label: label,
                    data: avgData,
                    borderWidth: 1,
                    fill: true,
                    spanGaps: true,
                    lineTension: 0.2,
                    backgroundColor: color,
                    borderColor: '#3A4250',
                    pointRadius: 2
                }
                    ]
            },
            options: {
                    responsive: true,
                    scales: {
                xAxes: [
                    {
                    type: 'time',
                    distribution: 'series',
                    ticks: {
                        source: 'labels'
                    }
                    }
                ],
                yAxes: [
                    {
                    scaleLabel: {
                        display: true,
                        labelString: label
                    },
                    ticks: {
                        stepSize: 0.5
                    }
                    }
                ]
                    }
            }
            });
            
            const progressEl = document.getElementById(el + '_progress');
            progressEl.remove();
            
            try {
                let app = firebase.app();
                let features = ['auth', 'database', 'messaging', 'storage'].filter(feature => typeof app[feature] === 'function');
                document.getElementById('load').innerHTML = `Firebase SDK loaded with ${features.join(', ')}`;
            } catch (e) {
                console.error(e);
                document.getElementById('load').innerHTML = 'Error loading the Firebase SDK, check the console.';
            }
            
        }

      Deploy

      firebase deploy

      Expected results and debugging

      The WebApp should look similar as shown in the figure below

      WebApp on a Nexus 5X

      Credits

      [1] Talk to your CD player using Google Home
      Sander Vonk, 2017-Oct-15, retrieved 2018-02-16.
      coertvonk.com/sw/embedded/google-home-ifttt-esp8266-integration-23066
      [2] Itead Sonoff S20 + MQTT + Google Assistant
      Coert Vonk, 2018-Mar-01, retrieved 2018-Mar-01.
      github.com/cvonk/esp8285-mqtt_light
      [3] Build a Weather Station using Google Cloud IoT Core and MongooseOS
      Alvaro Viebrantz, 2017-Oct-15, retrieved 2018-Feb-16.
      medium.com/google-cloud/build-a-weather-station-using-google-cloud-iot-core-and-mongooseos-7a78b69822c5
      [4] Overview of Internet of Things
      Google, 2017-Sep-28, retrieved 2018-Mar-17.
      cloud.google.com/solutions/iot-overview

      create and register device by hand

      Ameba : Getting Started With Google Cloud IoT

      Google Assistant integration: https://codelabs.developers.google.com/codelabs/assistant-codelab/index.html?index=..%2F..%2Findex#0

Introduction

Owning a backyard pool means learning about chemicals, and dealing with covers, vacuuming, brushes and adjusting the thermostats and circuits. A saltwater pool makes things a bit easier, but you still need to keep an good eye on the system.

The OPNpool integrates the functionality of a traditional Pool Controller into the modern smart home. It keeps tabs on the status of the connected controller, pool pump and chlorinator. This provides not only a more convenient solution than physically interacting with the pool equipment, but the ability to create automations that runs the pump for a duration depending on the temperature.

Thanks to the people that reverse engineered RS485-based protocol, OPNpool can listen into the communication between the pool components. The ESP32 shares the resulting state information as JSON/HTTP and can publish it using MQTT. Last but not least, it integrates seamlessly with Home Assistant thanks to MQTT Discovery.

Credentials are provisioned using an Android app. It even supports over-the-air updates when you decide to hack the software!

OPNpool interface

Features:

  • Visualizes the status of the thermostats, pump, chlorinator, circuits, and schedules.
  • Allows you adjust the thermostats and toggle circuits
  • No physical connection to your LAN
  • Supports over-the-air updates
  • Easily one-time provisioning from an Android phone
  • Integrates with MQTT and Home Assistant
  • Accessible as a webapp
  • Protected with IP68 waterproof case and connectors
  • Does not require a power adapter
  • Open source!

This device was tested with the Pentair SunTouch controller with firmware 2.080, connected to an IntelliFlo pump and IntelliChlor salt water chlorinator.

This open source and hardware project is intended to comply with the October 2016 exemption to the Digital Millennium Copyright Act allowing “‘good-faith’ testing,” in a controlled environment designed to avoid any harm to individuals or to the public.

This open source project is hosted at

Continue reading to learn about the RS-485 bus access.

Copyright © 1996-2022 Coert Vonk, All Rights Reserved