Google Assistant switches Sonoff S20

Switch your light on/off using your voice and the help of Google Assistant. Sure, you can run to the store and purchase a preconfigured light switch, but what’s the fun in that and more importantly these switches with their close-source software require access to your home network.

Achieving this simple level of automation took more doing than I expected. By sharing my notes, I hope you can clear the hurdles with ease.

Goal

simplified flow diagram
Simplified flow diagram
Triggered with the phrase “Hey Google”, the Google Assistant is able to feed the words that follow into its database of applets, one of which we will create. Once the applet recognizes the instruction, it sends a command towards our Sonoff smart outlet. Details of this conversation are something like the one shown in the diagram below.

Design

We use a $10 Sonoff smart outlet and change the firmware so it can be controlled using MQTT. To connect Google Assistant to MQTT, we use Kappelt’s gBridge.

Sonoff S20
Sonoff S20

Required hardware:

  • Google Assistant enabled device, such as a Google Home or Android Phone.
  • Sonoff S20 smart outlet connected to e.g. a lamp (any model supported by Tasmota will do),
  • a router that can serve as a reverse proxy such as DD-WRT,
  • a lightweight computer such as the Raspberry Pi 3.

data flow diagram
Data flow diagram (click to enlarge)

Implementation

We will work our way up starting with the Smart Switch all the way to Google Assistant.

Make Sonoff S20 respond to messages (Tasmota)

The key ingredient of making the Sonoff S20 work with Google Assistant is the MQTT protocol. The abbreviation MQTT stands for Message Queue Telemetry Transport, where the keyword is telemetry, or remote monitoring. By creating topics, you can enable different parts of your application to subscribe to specific streams of data. The Tasmota firmware includes this MQTT protocol.

Tasmato message flow
Tasmato message flow

The process of flashing new firmware is described in videos such on youtube.com and Theo Arend’s Wiki. In turn, my short summary:

Flash the firmware

Here we use the Python based flash tool.

Sonoff S20, FTDI Friend, lab power supply
Sonoff S20, FTDI Friend, lab power supply

Follow the steps below.

  1. Make sure Python 2.7 is installed on your computer
  2. Flash tool
    • Download esptool and expand the archive.
    • Install the dependencies: python setup.py install
  3. Connect serial port
    • Unplug the Sonoff S20 from the outlet, take the cover off and connect it to your computer using a USB/Serial adapter. Make sure the USB/Serial adapter can supply enough current to the ESP SoC.
    • Note the serial port that was your operating system assigned (e.g. COM10).
    • Counterintuitive, I had to connect RX-RX and TX-TX.
  4. Tasmota firmware
    • Download the Tasmota firmware sonoff.bin
    • Put the Sonoff S20 in flash mode by holding down the push button while connecting the serial port (and hence its power supply).
    • Clear the flash on the Sonoff S20: esptool.py --port COM10 erase_flash
    • Put the Sonoff S20 back in flash mode.
    • Flash the firmware: esptool.py --port COM10 write_flash -fs 1MB -fm dout 0x0 sonoff.bin
    • Disconnect and reassemble the Sonoff S20.

Boot

The Wifi Credentials for the Tasmota firmware are configured by putting it in Access Point mode.

  1. Plug it into the wall and press its button 5 times.
  2. On a mobile device, connect to ESP_xxxx via WiFi. Make sure the device stays connected (some will auto-connect to a different network if no internet connection is detected).
  3. Configure the Sonoff S20 by connecting to its AP WiFi Network, and use a browser to connect to http://192.168.4.1
  4. Setup the Sonoff S20 by scanning for Wifi networks, and configuring its credentials. While you are at it, set the hostname (e.g. sonoff-socket1). Save and wait for it to reboot.
  5. Connect your mobile device back to your regular WiFi network.
    • Configure

      By now the Sonoff S20 should have connected to your WiFi. We connect to its web interface and finish the configuration.

      1. Open your browser at the IP name/address assigned, http://sonoff-socket1
      2. Test by pressing Toggle what should switch the relay on/off.
      3. Configure module
        • Configuration > Configure Module > Module type = Sonoff S2X
        • Save and wait for reboot
      4. Configure MQTT
        • host = IP name/address of your Raspberry Pi
        • port = 1883
        • topic = sonoff/socket1
        • full topic = %prefix%/%topic%/
        • Save and wait for reboot

    MQTT Message Broker (Mosquitto)

    As mentioned before, the key protocol is MQTT. In this example, the message runs on a Raspberry Pi on our LAN).

    Mosquitto message flow
    Mosquitto message flow

    To start, install the Eclipse Mosquitto implementation:

    sudo apt update
    sudo apt install -y mosquitto mosquitto-clients
    sudo systemctl enable mosquitto.service
    sudo systemctl start mosquitto.service
    netstat -lt  # ensure it listens on port 1883

    Test to see if the MQTT broker is listening on port 1883, using netstat -lt. When you listen to all topics (mosquitto_sub -t '#' -v), you should see a periodic status report from the Sonoff S20, such as tele/sonoff/socket1/LWT Online.

    Control the relay on the Sonoff S20, by publishing values to the topic cmnd/sonoff/socket1/power

    • mosquitto_pub -t 'cmnd/sonoff/socket1/power' -m 1
    • mosquitto_pub -t 'cmnd/sonoff/socket1/power' -m 0

    At this point the roads diverge. The straight and easy path uses the hosted service gbridge.io. You would continue with their Getting Started guide and be done. The other choice is to host the MQTT/Assistant bridge yourself as described in the remainder of this document.

    This other path is windy but might be more interesting. It involves hosting the MQTT/Assistant bridge. This approach is loosely based on Self-Hosted gBridge, with various changes based on e.g. community contributions. Without further adieu, the self-hosted bridge is described in the remainder of this document and requires:

    • A router you have control over, and can function as a reverse proxy (e.g. DD-WRT).
    • A public domain name for your router
    • An SSL certificate for the router (e.g. LetsEncrypt)
    • A computer that is always on (e.g. Raspberry Pi 3)
    • Traversing the Router

      Google Assistant accesses our MQTT server through our router using HTTPS. This router therefore must have a public DNS name and a certificate for HTTPS and support reverse proxy. The reverse proxy translates between Google Assistant’s HTTPS and HTTP on our LAN. Instead of the reverse proxy, it might be possible to forward specific ports and do the HTTPS/HTTP conversion on the Raspberry Pi.

      Router message flow
      Router message flow

      For this example, we assume a router with DD-WRT firmware that includes dnsmasq and pound.

      Public DNS name

      Start by giving your router a public DNS name. Google Assistant uses this name when sending commands towards our MQTT server. In this example, we make a subdomain under an existing web domain. Another approach would be to use a DDNS service.

      At your DNS provider, use their DNS Zone Editor to add the subdomain mqtt

      mqtt.domainname.com 1440 IN A your_rtr_public_ip_addr
      This may take up to 72 hours to propagate. If you haven’t done so already, convert your domain’s certificate to a wildcard certificate and download it. The wild card certificate can be used for of your subdomains.

      Certificate for HTTPS

      For Google Assistant to be able verify the identity of your MQTT server we need to install the wildcard certificate that we downloaded in the previous step. Concatenate the private key, certificate and CA certificate into /jffs/etc/pound/yourdomain.pem

      Reverse proxy

      As said, the reverse proxy translates between Google Assistant’s HTTPS and HTTP on our LAN. This way only your router needs to understand HTTPS. We will configure it so, that when the router receives an HTTPS (or HTTP) request for URL https://mqtt.yourdomain.com/gapi, it will forward it as HTTP to the Raspberry Pi.

      For the reverse propy we’ll loosly follow DD-WRT Reverse Proxy and HTTPS.

      To get up and running, we need the binaries, a script that starts it and configuration files. These files are available through this GitHub page. The scripts are based on Frater’s post.

      1. Enable JFFS partition. From the router’s GUI, enable JFFS (Administration » Management). On first use, check the box to “clean internal flash storage”. Reboot as needed.
      2. Configure DNSMasq on your router, to always give the same name and IP address (based on MAC) to the Sonoff S20 and Raspberry Pi. This example assume these names are sonoff-socket1 and mqtt.
      3. Pound binaries. Our router’s firmware didn’t include pound binaries, so we copied it from an older firmware (kongac’s r33010M)
        scp rtr2:/usr/sbin/pound rtr2:usr/sbin/poundctl /jffs/sbin/
        scp rtr2:/usr/lib/libssl.so.1.0.0 rtr2:/usr/lib/libcrypto.so.1.0.0 /jffs/lib
      4. Install startup script. Copy the script /jffs/sbin/pound.sh and its helper write_pound_cfg from GitHub.
      5. Configure ports to listen on. The file /jffs/etc/pound/pound.pt1 specifies what ports to listen at. The example below uses port 80 and 443, so make sure they are not already in use for the DD-WRT UI (change dd-wrt web admin port).
        TimeOut         120
        Alive           30
        Control         "/tmp/pound.ctl"
        
        ListenHTTPS
                Address 0.0.0.0
                Port 443
                xHTTP 1
                Cert "/jffs/etc/pound/yourdomain.pem"
      6. Configure forward rules. The file /jffs/etc/pound/pound.pt2 specifies where to forward the HTTP request to.
        Service "mqtt"
                        HeadRequire "Host:.mqtt.yourdomain.com.*"
                        Url "/gapi.*"
                        BackEnd
                                Address ip_address_of_your_rpi
                                Port 8080
                        End
                End
        End
        ListenHTTP
                Address 0.0.0.0
                Port 80
                xHTTP 1
                Service "mqtt"
                        HeadRequire "Host:.mqtt.yourdomain.com.*"
                        Url "/gapi.*"
                        BackEnd
                                Address ip_address_of_your_rpi
                                Port 8080
                        End
                End
        End
      7. Start pound using /jffs/sbin/pound.sh start
      8. Pointing a web browser to either
        • https://mqtt.yourdomainhome.com/gapi
        • http://mqtt.yourdomainhome.com/gapi
        Since we haven’t setup the HTTP/MQTT bridging software, you will not get the page requested. The name should however resolve and eventually you’ll get The service is not available. Please try again later.

      Adding a skill to Google Assistant

      Before we can setup the HTTP/MQTT bridge, we need to generate some identifiers on Google Cloud. To do so, we roughly follow the guide from Kappelt’s Self-Hosted gBridge.

      Google Assistant message flow
      Google Assistant message flow

      To get started, head over Google Actions Console using the same account that you use for Google Home.

        • Project name = gBridge
        • Development experience = Smart Home » Smart Home
        • Name your Smart Home action
          • Display name = whatever it accepts is fine. Save
          • Return to previous web page
        • Setup account linking
          • No, I only want to allow account creation on my website Next
          • Linking type = OAuth, Implicit Next
        • OAuth Client Information
          • Client ID issued by your Action to Google = any random id that you generate, write down this Account Linking Client ID
          • Authorization URL = https://mqtt.yourdomain.com/gapi/auth
        • Configure your client (optional)
          • Skip this Next
        • Testing instructions
          • Skip this Save
        • Add action
          • Fulfillment = https://mqtt.yourdomain.com/gapi Save
        • Set your Language

      Continue on Google Cloud with the following steps

        • Select your project (from the All list)
        • Project ID = write down this Google Project ID
        • + ENABLE APS AND SERVICES
        • HomeGraph API
        • ENABLE
        • Create credentials » API Key
        • Your API key = write down this HomeGraph API Hey

      Bridging Google Assistant and MQTT (gBridge)

      Time to install the MQTT/HTTP bridge. Here we use a self hosted Kappelt gBridge. This consist of docker packages that will run on the Raspberry Pi.

      gBridge message flow
      gBridge message flow

      Start with installing Docker

      sudo apt-get install libffi-dev pwgen tzdata 
      sudo pip install docker-compose

      Create a /opt/gbridge/docker-compose.yum, substituting your own , , and .

      version: '3'
      networks:
         backend:
            driver: bridge
         web_frontend:
            driver: bridge
      services:
         database:
            image: 'yobasystems/alpine-mariadb:latest'
            restart: always
            environment:
               MYSQL_RANDOM_ROOT_PASSWORD: 'true'
               MYSQL_ROOT_PASSWORD: <your_mysql_passwd>
               MYSQL_DATABASE: gbridge_db
               MYSQL_USER: gbridge_db
               MYSQL_PASSWORD: <your_mysql_passwd>
            expose:
               - '3306'
            networks:
               - backend
         cache:
            image: 'redis:4'
            restart: always
            expose:
               - '6379'
            networks:
               - backend
         web:
            image: 'pkap/gbridge-web-nginx:arm32v6-latest'
            restart: always
            ports:
               - '8080:80'
               - '443:443'
            environment: &webapp-environment
               APP_ENV: production
               APP_KEY: 'base64:NTYyZWE3NThjZDYxNzM3Nzg2ZTM2MGQ3NDY5MjY0YTI='
               APP_DEBUG: 'false'
               APP_LOG_LEVEL: warning
               APP_URL: 'http://localhost'
               DB_CONNECTION: mysql
               DB_HOST: database
               DB_PORT: 3306
               DB_DATABASE: gbridge_db
               DB_USERNAME: gbridge_db
               DB_PASSWORD: <your_mysql_passwd>
               BROADCAST_DRIVER: log
               CACHE_DRIVER: file
               SESSION_DRIVER: file
               SESSION_LIFETIME: 120
               QUEUE_DRIVER: sync
               REDIS_HOST: cache
               REDIS_PASSWORD: 'null'
               REDIS_PORT: '6379'
               MAIL_DRIVER: smtp
               MAIL_HOST: ERROR
               MAIL_PORT: ERROR
               MAIL_USERNAME: ERROR
               MAIL_PASSWORD: ERROR
               MAIL_ENCRYPTION: ERROR
               GOOGLE_CLIENTID: <your_google_clientid>
               GOOGLE_PROJECTID: <your_google_projectid>
            links:
               - database
               - cache
               - web-fpm
            depends_on:
               - database
               - cache
               - web-fpm
            networks:
               - web_frontend
               - backend
            volumes:
               - websrc:/var/www
         web-fpm:
            image: pkap/gbridge-web-fpm:arm32v6-latest
            restart: always
            networks:
               - backend
            volumes:
               - websrc:/var/www
            environment: *webapp-environment
         redis-worker:
            image: 'pkap/gbridge-redis-worker:arm32v6-latest'
            restart: always
            environment:
               GBRIDGE_REDISWORKER_REDIS: 'redis://cache:6379'
               GBRIDGE_REDISWORKER_MQTT: 'mqtt://<your_rpi>:1883'
               GBRIDGE_REDISWORKER_MQTTUSER: ""
               GBRIDGE_REDISWORKER_MQTTPASSWORD: ""
               GBRIDGE_REDISWORKER_HOMEGRAPHKEY: <your_gbridge_redisworker_homegraphkey>
            networks:
               - backend
            links:
               - cache
            depends_on:
               - cache
      volumes:
         websrc:

      If you tried this before but failed, I suggest to start from scratch (per https://community.gbridge.io/t/nginx-config-for-rpi-self-hosted/115/11)

      sudo su
      cd /opt/gbridge
      docker-compose rm -f web redis-worker web-fpm database cache*
      docker volume rm gbridge_websrc
      reboot

      Continue with a clean slate
      sudo su
      cd /opt/gbridge
      docker system prune -a
      apt-get update
      apt-get upgrade

      Clear the cookies for the site in your browser (I found this the most common source of generic gBridge error messages in the browser)

      Bring up the containers docker-compose up : : database_1 | 2019-09-07 3:40:23 0 [Note] Reading of all Master_info entries succeeded database_1 | 2019-09-07 3:40:23 0 [Note] Added new Master_info ” to hash table database_1 | 2019-09-07 3:40:23 0 [Note] /usr/bin/mysqld: ready for connections. database_1 | Version: ‘10.3.17-MariaDB’ socket: ‘/run/mysqld/mysqld.sock’ port: 3306 MariaDB Server

      Meanwhile in a browser, access mqtt.vonk:8080/ should show the login page

      ^c out of docker-compose up, and instead start it detached using docker-compose up --detach

      Setup user and password using docker-compose exec web-fpm php artisan migrate. Should reply with a password and a bunch of Migrating *table messages.

      Login from browser, in my case at http://mqtt.vonk:8080 gives the login page. Login with and passwd 123456, don’t remember me.

        • Use a password with at least one number and at least one special char.
        • Create a device
          • Name = soft light
          • Device Type = Light
          • Traits = Select Supported Traits, On and Off
          • + Add

      MQTT forward

      The MQTT topics used by the Tasmota firmware and the gBridge don’t match. Solve this by

      • forwarding traffic for gBridge/u1/# to cmnd/, and
      • from stat/# to gBridge/u1/.
      Here “gBridge/u1” is the the user id of your gBridge account as listed in the http://mqtt:8080/profile.

      Mosquitto message flow
      Mosquitto message flow

      To do so, on the Raspberry Pi, create /etc/mosquitto/conf.d/gbridge.conf as

      connection gBridge
      address 127.0.0.1:1883
      topic cmnd/# in 0 "" gBridge/u1/
      topic stat/# out 0 "" gBridge/u1/
      Restart the MQTT services, sudo service mosquitto restart.

      Log the messages, mosquitto_sub -v -t '#'.

      Add the device to Google Home Hub

      The final step is done on a mobile device using the Google Home App.

        • Set up a new device.
          • Works with Google
          • Have something already set up?
          • Select the Google Display Name for your project (that is prefixed with [test]

Embedded software developer
Passionately curious and stubbornly persistent. Enjoys to inspire and consult with others to exchange the poetry of logical ideas.

4 Replies to “Google Assistant switches Sonoff S20”

  1. Hello Coert,

    nice tut. I have a problem connecting my gbridge with the home app. It tells me it can’t update the settings and I should check my internet connection. The page is reachable.

    I have no clue what the problem might be.

  2. Hey Coert,

    First, let me thank you for saving me a lot of heartache. I was trying to implement something similar to your setup and was going through the pains of identifying the packages that I need to refer to in the docker-compose.yaml for my gBridge host (also a Pi3). Your writeup solved the last few issues I had still standing.

    I have a quick question regarding the mapping of MQQT topics between gBridge and Tasmota. Everywhere I read about these mappings it seems to be in the context of MQTT broker bridging. Your writeup suggests that maping also works to translate accross clients (i.e. gBridge and Tasmota). Am I reading your article correctly, or did you install an MQTT server on the gBridge RPi, and bridged it with anothe MQTT server running a separate RPi?

    Thanks!

    -LuisB

  3. #LuisB, you read it correctly. Usually, the mechanism is to forward messages to another broker. In my case, I use it to change the topic between the Tasmota and the gBridge.

    Happy to hear you found it useful,
    /c

  4. Ok, thanks much.

    One more question – perhaps you ran into this. It all works great from Google to MQQT, but I can’t make it go the other way for updating the state of the switch. If I turn off the switch by posting my own MQTT message, the switch turns off and I see the gBridge topic getting updates with the new state of the switch, but google seems to not update its own state (i.e. the assistant still thinks the switch is off). The gBridge command topic is set to gBridge/u1/d1/onoff and the state topic is gBridge/u1/d1/onoff/set, so that looks pretty good. It’s probably something on my end, but debugging this is not trivial.

Leave a Reply

Your email address will not be published. Required fields are marked *

 

This site uses Akismet to reduce spam. Learn how your comment data is processed.