Before diving into the details, let me emphasize, that the key mechanism is MQTT. Most home automation systems support MQTT. Details about configuring MQTT for other systems than Home Assistant are given later in this chapter.
We will focus on Home Assistant with an install base of over 100,000 user
An open source home automation that puts local control and privacy first. Powered by a worldwide community of tinkerers and DIY enthusiasts. Perfect to run on a Raspberry Pi or a local server.
Home Assistant
We auto populate the Home Assistant entities using MQTT Discovery. This enables us to use MQTT devices with only minimal configuration effort on the side of Home Assistant. A few of these messages are shown in the section MQTT device discovery of chapter Interface.
Home Assistant supports will use the messages and automatically populate the entities shown below.
Device entities on Home Assistant
Device entities on Home Assistant cont’d
Dashboard
In the Home Assistant configuration.yaml we add some rules and template sensors as shown in hassio/packages. This also includes some rules that we use for the PoolMath integration.
The Lovelace dashboard tab includes several cards as shown below. The YAML code can be found in hassio/lovelace.
Idle
When the pump is not running, the cards shown below wil be visible.
When for instance the pool circuit is activated, additional cards show up.
Other home automation systems
The OPNpool device publishes state transitions using MQTT. You find an overview of the topics in the table below.
When the OPNpool device connects to the WiFi access point, it will be assigned an IP address. Only the access point and the device know this address. Other hosts can access the device by its mDNS name opnpool.local.
This relies on service auto-discovery using the Multicast DNS (mDNS) protocol. Note that mDNS’ use of multicast packets is designed to work within a single IP subnet. That implies that in general your computer/phone need to be on the same subnet as the OPNpool device for the mDNS protocol to work.
HTML
As you may have seem in the video in the Deploying chapter, the OPNpool device can be controlled through a simple HTML UI. This allows the user to monitor the state of the pool controller, pump and chlorinator.
This UI is accessed through the URL http://opnpool.local/. The unsecure connection implies that security solitary relies on your WiFi credentials. If you want to access the device from the public internet, I urge you by tunneling into your LAN instead of simply opening a port on your router.
To conserve memory, the OPNpool device only contains the minimum HTML code that refers to an external site for the Stylesheet and the replacement element that refers to the JavaScript.
Notes on going dark
To switch to a dark theme:
Change data-theme="a" data-content-theme="a" to data-theme="b" data-content-theme="b" in replace-body.js, and
Change .rs-bg-color from white to #2b2b2b in index.css.
data-theme in the HTML , and
Some examples of the UI are shown below
Request activate Pool circuit
Air temperature
Pool thermostat
Pump speed
Salt level
Schedules
JSON
The raw state of the pool controller is accessible through http://opnpool.local/json. The HTML page uses this internally, but you can also poll this state from home automation systems. However, if your home automation supports MQTT, I suggest using MQTT as described in the Home Assistant chapter.
The interface image makes the device an “OPNpool Interface”. Besides monitoring the pool controller and providing access to a web interface, it provides automatic updates, network discovery and supports post mortem debugging.
Architecture
The entry point of the code is app_main() in the file main/main.c.
app_main() starts by initializing devices and connecting to the WiFi network. The interface appplication is split over different tasks that communicate over message queues offered by the Free Real Time Operating System (Freitas).
factory_reset_task
pool_task
ota_update_task
http server
mqtt_task
hass_task
factory_reset_task
A factory reset erases the Wifi credentials and returns to the factory app. This allows the device to be provisioned from scratch. To perform a Factory Reset, pull GPIO#0 low for five seconds, while the interface app is running (not during boot up). The development board, has a boot button) that services the same function. I
I (753) factory_reset_task: Waiting for 3 sec BOOT/RESET press ..
W (26408) factory_reset_task: Factory reset ..
The pool_task forms the heart of OPNpool. It handles data received from the pool controller, and can send requests to the pool controller.
Its responsibilities are:
Initially, it queues a few messages that request thermostat and scheduling information from the pool controller.
If it receives a message in the to_pool_q, it creates a corresponding network message. It then calls datalink_tx_pkt_queue to add the preamble and tail, and queues it for transmission through the rs485 driver.
Reads bytes from the rs485 driver to create a data packet. It passes this up the stack create a network message. Based on that it updates the pool state information and publishes it using the MQTT task.
The protocol stack is shown in the illustration below.
Pool task layers
The following sections provide more information about the protocol stack.
RS-485 driver
Starting from the physical RS-485 bus, the signal passes through the MAX3485 transceiver that connects to the RX, TX and RTS pins of the ESP32 UART. This UART presents the signal as bytes to the driver.
The driver:
Initializes the UART.
Provides functions to read and write bytes.
Bytes to be transmitted are queued as as rs485_q_msg in rs485->tx_q. When a transmit opportunity arrises, the pool_task dequeues from rs485->tx_q, relies on the driver to transmits the bytes.
Transmitting on the RS-485 is a bit tricky because it operates in half-duplex and has multiple nodes on the same single twisted pair. In practice, the pool controller appears to only accept messages after it broadcasts its periodic status message, and before it communicates with the chlorinator. Therefore, outgoing bytes are queued, until the moment arrives where they can be transmitted
It communicates with the datalink layer using the rs485_handle_t device handle.
typedef struct rs485_instance_t {
rs485_available_fnc_t available; // bytes available in rx buffer
rs485_read_bytes_fnc_t read_bytes; // read bytes from rx buffer
rs485_write_bytes_fnc_t write_bytes; // write bytes to tx buffer
rs485_flush_fnc_t flush; // wait until all bytes are transmitted
rx485_tx_mode_fnc_t tx_mode; // controls RTS pin (for half-duplex)
rs485_queue_fnc_t queue; // queue to handle->tx_q
rs485_dequeue_fnc_t dequeue; // dequeue from handle->tx_q
QueueHandle_t tx_q; // transmit queue
} rs485_instance_t;
typedef struct rs485_instance_t * rs485_handle_t;
Datalink layer
Provides framing and byte error detection. The interface with the network layers is through datalink_pkt_t.
It supports both the A5 and IC protocol.
It communicates with the network layer using the datalink_pkt_t structure.
typedef struct datalink_pkt_t {
datalink_prot_t prot; // datalink_prot as detected by `_read_head`
uint8_t prot_typ; // from datalink_hdr_a5->typ
uint8_t src; // from datalink_hdr_a5->src
uint8_t dst; // from datalink_hdr_a5->dst
datalink_data_t * data;
size_t data_len;
skb_handle_t skb;
} datalink_pkt_t;
Its functions are:
datalink_rx_pkt(), receives bytes from the RS-485 bus and makes a datalink_pkt from it. It is implemented as a state machine, using socket buffers to store the bytes received.
datalink_tx_pkt(), receives a datalink_pkt and wraps it with a header and tailer. It then queues it for transmission on the RS-485 bus. Once there is a transmit opportunity, the pool_task send it using the rs485 driver.
Network layer
This layer translates between datalink_pkt and the protocol-agnostic network_msg.
It communicates with the `poolstate` layer using the network_msg_t structure.
network_rx_msg(), translates a datalink_pkt writes it as network_msg.
network_create_msg(), creates a datalink_pkt from a network_msg.
Poolstate
The sysState_t structure stores the state of the pool system. It is updated based on status messages from the controller, and by querying the controller. Once the controller replied to these queries, pool() starts honoring commands from the Serial port connected to the computer.
ota_update_task
This is the same git submodule that we used for Factory application earlier.
Each time the device powers up, it checks to see if an new interface app is available. If so, it will download the image and switches to the new version. [doc] To trigger an OTA update, power cycle the device, or press the EN button.
When the new interface app has critical errors, a rollback mechanism keep the device working by automatically rolling back to the previous working application.
After the app is downloaded using OTA, it is marked as pending_verify. Once the application determines that it is working well, it marks it as valid. On the other hand, when the application marks it as invalid or the device resets, the bootloader will boot the previous working application. [doc]
http server
Albeit not technically a task, we’ll discuss it here.
The HTTP server provides an UI to see the pool state and control a few settings.
The main_task uses the git module wifi_connect to establish a connection to the WiFi access point. Once established, it calls httpd_register_handlers to register the URI handlers for incoming GET requests.
These handlers are:
httpd_root, replies to GET /
httpd_ico“, replies to GET /favicon.ico
httpd_json, replies to GET /json
httpd_root()
To save memory space on the ESP32, the device returns a minimum HTML page. The body part is replaced by the replace_body.js script.
httpd_ico()
The browser places the favicon next to the page’s title on the tab. This function simply sends an HTTP response containing the favicon.
httpd_json()
The Arduino replies to any command with the pool state information formatted as JSON. See illustration further down.
If query variables were passed, they will be send to the ipc->to_pool_q. The pool_task will create a corresponding message and send it to the pool controller.
For the switches, it supports the values ON and OFF. For a thermostat (climate) mode, it supports the values off, heat, solar_pref and solar. For the thermostat temperature it accept a value in Fahrenheit.
Example
GET /json request
GET /json response
For details on web integration, refer to the Web UI section.
mqtt_task
IBM’s MQ telemetry transport (MQTT) is a lightweight, publish-subscribe network protocol that transports messages between devices. The keyword is telemetry, or remote monitoring.
MQTT is rapidly becoming the prevailed protocols to connect large numbers of devices. The mqtt_task implements an MQTT client.
Responsibilities of the task:
Connect to MQTT broker.
Initially, check if there is a coredump in flash memory, then publish it to MQTT.
Subscribe to control topics, for diagnostic commands.
Subscribing or publishing on MQTT topics on behalf of other processes.
Connect to MQTT broker
The code calls esp_mqtt_client_init and waits until the MQTT connection with the broker is established. It learns about a successful connection when it the callback function receives a MQTT_EVENT_CONNECTED.
If the connection were to be broken, the ESP IDF SDK will automatically reconnect to the broker.
Coredump to MQTT
As the devices that are deployed in the field, they have no serial monitor connected. Instead, we configure the ESP-IDF so the coredump is written to flash memory. Next time the ESP32 boots up, and it finds a coredump in flash, it uses the git submodule ESP32_coredump-to-server to publish the coredump to MQTT under the topic opnpool/data/coredump/ID.
Subscribe to control topics
Initially, the device asks MQTT task to subscribe to the topics listed below. Here ID is an unique device identifier.
opnpool/ctrl
opnpool/ctrl/ID
When another devices publishes a message on these topics, the MQTT task will handle the message. It supports the following values:
who, request device info such as device id, firmware version, WiFi details, MQTT details and heap
restart, restarts the device
htstart, starts heap trace (when enabled)
htstop, stops heap trace (when enabled)
When the MQTT task receives a message on these topics, it will take action and publish a reply to pool/data/ID/ as shown below.
Publish “who” to “opnpool/ctrl”
Response of publishing “who” to “opnpool/ctrl”
Subscribe/publish on behalf of others
The hass_task queues a message to ipc->to_mqtt_q in order to subscribe or publish on a topic. The mqtt_task forwards this request to the ESP-IDF SDK.
When subscribing, the topic is also added to the _subscribers list, in case the connection with the broker needs to get re-established.
When another MQTT client publishes on a topic that another process subscribed to, the message is forwarded to the ipc->to_pool_q.
hass_task
Home Assistant is an open source home automation that puts local control and privacy first.
The hass_task’s duties are
Listen for commands over MQTT.
Support MQTT device discovery.
Listen for commands over MQTT
Initially, the device subscribes to the MQTT topics that lets other MQTT clients change pool controller settings such as thermostat set point or heating mode.
When another devices publishes a message on these topics, the MQTT task will forward forward it as a network message to the Pool task. The latter will call hass_tx_pair to handle the message.
For the switches, it supports the values ON and OFF. For a thermostat (climate) mode, it supports the values off, heat, solar_pref and solar. For the thermostat temperature it accept a value in Fahrenheit.
MQTT device discovery
We auto populate the Home Assistant entities using MQTT Device Discovery. This enables us to use MQTT devices with only minimal configuration effort on the side of Home Assistant.
Every five minute, the device publishes MQTT discovery messages, to announce its presence to Home Assistant.
Some examples of these configuration topics are:
topic = homeassistant/sensor/opnpool/air_temp/config;
value = {
"~": "homeassistant/sensor/opnpool/air_temp",
"name": "Pool air temp",
"stat_t": "~/state",
"unit_of_meas": "°F"
}
Rolling out a device to multiple sites introduces challenges. The WiFi credentials can no longer be part of the source code. Applying firmware updates or accessing crash information over USB becomes cumbersome. Even simply finding the IP address of the device on the LAN may be challenging.
To address such challenges we use a three tiered approach, in which the bootloader image decides what image to boot.
As usual, the bootloader image does some minimum initializations. If it finds a valid ota image, it passes control over to that image. If not, it starts the factory image.
The factory image takes care of provisioning WiFi and MQTT credentials with the help of a phone app. These credentials are stored in the nvs partition. It then downloads the ota image.
We refer to the ota image as the interface, as it provides the core of the functionality of the OPNpool device.
To facilitate this the flash memory is partitioned as shown below.
The partition table is shown in the table below.
start
name
type
subtype
0x001000
bootloader
0x009000
nvs
data
nvs
0x00d000
otadata
data
ota
0x00f000
phy_init
data
phy
0x010000
factory
app
factory
0x160000
interface_0
app
ota_0
0x260000
interface_1
app
ota_1
0x360000
coredump
data
coredump
Partition table
The following sections describes the process in further detail. A whole chapter is dedicated to the interface image, which provides the core functionality of the OPNpool device.
Bootloader image
A few more words on the bootloader:
If otadata partition is erased, it starts the factory image (assuming it is present).
After the first OTA update, the ota_data partition is updated to specify which OTA app slot partition should be booted next.
If the image works fine, it marks itself as ESP_OTA_IMG_VALID in the ota_data. Otherwise, it marks itself ESP_OTA_IMG_INVALID. This mechanism support image rollback to keep the device working after the update. It automatically rolls back to the previous version, if the image doesn’t pass its self test.
Factory image
The factory image, handles the provisioning and triggers an over-the-air (OTA) download of the interface image. It then reboots, the bootloader will start the interface image.
These steps are described in detail in the following section.
Provisioning WiFi credentials
For testing purposes, WIFI_CONNECT_SSID, WIFI_CONNECT_PASSWORD and OPNPOOL_MQTT_URL can also be provisioned using Kconfig. If empty, the device will use credentials from flash memory.
A phone is used to connect to the factory app using Bluetooth Low Energy (BLE). The factory image will advertise itself to the phone app. Using this phone the user specifies the WiFi and MQTT credentials. During the process the phone remains in contact with the ESP while the WiFi is set up and connects to an access point. The WiFi and MQTT credentials are stored in the nvs partition of flash memory.
You can find the source code of this app in the android directory. If you have an iOS phone, or you have problems running the Android app, you can extend esp_prov.py to include mqtt_url similar to what is shown here“.
The figure and video below give an impression of the provisioning process
I (907) factory: Starting BLE provisioning
I (1338) ble_prov: advertising as "POOL_CC4504"
I (99768) ble_prov_handler: Received WiFi credentials:
ssid Guest Barn
password xxxxxxxxxx
I (100098) ble_prov_handler: WiFi Credentials Applied
I (100288) ble_prov_handler: Connecting ..
I (104328) factory: IP addr 10.1.1.118
I (104328) ble_prov: STA Got IP
I (104328) ota_task: Checking for OTA update (https://coertvonk.com/cvonk/pool/interface.bin)
I (104338) ota_task: Running from part "factory" (0x00010000)
I (104548) ota_task: Writing part ota_0 at offset 0x160000
I (104548) ota_task: Firmware on server: interface.f6d8367-dirty (Mar 2 2022 10:26:42)
I (104548) ota_task: Firmware running: factory.6bc4c65-dirty (Mar 2 2022 14:52:42)
W (104558) ota_task: Downloading OTA update ..
I (105848) ble_prov_handler: Connected state
I (109668) ota_task: Wrote 5% of 1280 kB
:
To establish the WiFi connection, the code base uses the git submodule ESP32_wifi-connect. This component makes this API available as a reusable component. It connects to a WiFi Access Point, and automatically reconnects when the connection drops.
Download the Interface image
The ESP-IDF API provides a mechanism that allows a device to update itself based on data received while the normal firmware is running. Our code base uses the git submodule ESP32_ota-update-task that makes the API available as a component.
The location of the factory image is specified by OTA_UPDATE_FIRMWARE_URL in Kconfig.
Note on HTTPS
To use an HTTPS connections, you will need to add the server’s public certificate to components/ota_update_task/CMakelists.txt, and uncomment some lines in ota_update_task.c with server_cert_pem.
The code checks for an OTA update on a network server. If the image is different, it will download it. Upon completion, the device resets to activate the downloaded code. Note that we use the term “update” loosely, because it can also be used to downgrade the firmware.
To determine if the currently running code is different as the code on the server, it compares the project name, version, date and time. Note that these are not always updated by the SDK. The best way to make sure they are updated is by committing your code to Git and building the project from scratch (by removing the build directory).
During debugging, you may want to flash the image image directly as part of the “IDF-SDK: Build, flash and start monitor your device” cycle. To prevent that image from being overwritten by an OTA update, the update mechanism is disabled when the interface image is loaded in the factory partition.
An example of the update process is shown in the figure below.
I (3222) ota_task: Checking for OTA update (https://coertvonk.com/cvonk/pool/interface.bin)
I (3232) ota_task: Running from part "factory" (0x00010000)
I (3412) ota_task: Writing part ota_0 at offset 0x160000
I (3422) ota_task: Firmware on server: interface.f6d8367-dirty (Mar 2 2022 10:26:42)
I (3422) ota_task: Firmware running: interface.6bc4c65 (Mar 2 2022 10:29:53)
W (3432) ota_task: Downloading OTA update ..
I (8142) ota_task: Wrote 5% of 1280 kB
:
I (23602) ota_task: Connection closed
I (24322) ota_task: Prepare to restart system!
OTA debug trace
More details of this component can be found at Github.
Describes the protocol that the pool controller uses on the RS-485 bus to communicate with its attached devices.
Key characteristics of this bus are:
A pool controller initiates a query addressed to a specific device. The device will respond by performing an action and then respond back to the controller.
It is an half-duplex connection, so only one system can transmit at the time. The controller will send a query and wait for a response, or times out when the device does not respond within a predefined period.
The OPNpool device sniffs this RS-485 bus to gather state information, and presents it to the user. The user can also change select parameters such as the thermostats or start e.g. the pool circuit.
The pool controller communicates with the pump and chlorinator over the RS-485 bus using a proprietary protocol. This protocol has been reverse engineered and documented in bits and pieces found on-line by people such as shown in the table below
The controller uses two different protocols to communicate with its devices:
“A5” messages are used to communicate with devices such as pumps. The first bytes are always 0x00, 0xFF, 0xA5
“IC” messages are used to communicate with the chlorinator. The first bytes are always 0x10, 0x02
A5 messages
The messages consist of a header, data and a 16-bit checksum. The length of the data block is specified in the header. The checksum is calculated starting at the last byte of the header (0xA5) up to the last data byte. Checksum is transmitted most significant byte first.
#define PACK8 __attribute__((aligned( __align__( uint8_t ) ), packed ))
typedef struct {
uint8_t pro; // protocol version
uint8_t dst; // destination
uint8_t src; // source
uint8_t typ; // message type
uint8_t len; // # of data bytes following
} PACK8 mHdr_a5_t;
The most significant 4 bits of source and destination address form the address group. The least significant 4 bits are the device number.
We further distinguish between messages to/from a pump and control messages. Examples of control messages are time, state, head and scheduling requests.
The IC messages consist of a header, data and a tail containing a 8-bit checksum and a 2 byte post-amble. The length of the data block depends on the message type specified in the header. The checksum is calculated starting at the beginning of the header up to the last data byte.
Describes methods for debugging the pool interface. Includes JTAG debugging and post-mortem coredump analysis.
Run time debug
Using `menuconfig`, the user can configure different levels to console logging. The verbosity is specified as None (0), Errors (1), Warnings and errors (2).
OPNPOOL_DBGLVL_DATALINK
OPNPOOL_DBGLVL_NETWORK
OPNPOOL_DBGLVL_POOLSTATE
OPNPOOL_DBGLVL_POOLTASK
OPNPOOL_DBGLVL_MQTTTASK
OPNPOOL_DBGLVL_HASSTASK
OPNPOOL_DBGLVL_HTTPD
Post mortem
The device includes support to generate core dumps on unrecoverable errors. This allows post-mortem analysis of the software state at the moment of failure. This way you can find what task, the instruction and what call stack lead to the crash. The core dump is written to the coredump partition in flash.
As we will see in the Interface chapter, the mqtt_task will read this coredump from flash and forward it over MQTT.
GDB over JTAG
The ESP32 supports the Open On-Chip Debugger. Say goodbye to ESP_LOG, and explore the world that was once the exclusive domain of in-circuit emulators. Using special pins on the ESP32, your computer can set break points, inspect variables and single step instructions.
OPNpool with ESP-PROG and computer
ESP32 board and JTAG adapter
The JTAG interface hooks directly to the CPU. That allows it do stop the CPU, set breakpoints and have access to whatever the CPU has access to.
The JTAG header is available on the OPNpool board. This 10 pin header should be connected to a JTAG/USB interface, such as the ESP-PROG. With the interface, make sure to set the JTAG PWR SEL jumper for 3.3V. Also try to keep the connection short (<10 cm).
Another option is to use the ESP-WROVER-KIT development board.
The ESP-WROVER-KIT has an integrated JTAG/USB interface what is available over the first of two USB channels. (The second channel is for the serial port.) If you go this route, make sure to connect the JTAG jumpers on JP2 to prevent Error: libusb_open() failed with LIBUSB_ERROR_NOT_SUPPORTED.
OpenOCD and driver
Under Windows, install the 64-bit FTDI D2xx Driver setup executable;
Using micro-USB cables, connect JTAG adapter and the serial port on the OPNpool board to the computer.
Wait until USB ports are recognized by Windows and drivers are installed. If they do not install automatically, then do so manually using the driver setup from FTDI.
The list of devices should contain two ESP-PROG specific USB entries with driver name “FTDIBUS (vxxxx)”:
Dual RS232-HS (Interface 0), connected to the JTAG port
Dual RS232-HS (Interface 1), connected to the UART port
Select Dual RS232-HS (Interface 0), and reinstall attached driver to the “WinUSB (v6xxxxx)”, see the picture below.
Windows Device Manager should no longer show the JTAG port, because WinUSB is a user-space driver.
Zadig to configure WinUSB driver.
Compile and debug
Set up the OpenOCD configuration
F1 » Select OpenOCD Board Configuration.
Choose the ESP32 chip (via ESP-PROG). (Unless you use the ESP-WROVER-KIT of course.)
Note that WROVER modules use a 1.8 SPI flash because of PSRAM limitations.
Others such as WROOM modules use 3.3V SPI flash. This flash voltage is important, because GPIO12 is shared with SPI flash. Selecting the wrong voltage may cause flash uploads to fail.
See if OpenOCD starts
F1 » ESP-IDF: OpenOCD Manager.
Choose Start OpenOCD.
Build the code with symbols and without optimalizations.
F1 » ESP-IDF: Launch GUI configuration tool
Specify the Debug (-Og) compiler optimalization level.
Press ^ed to build, upload and monitor over the serial port.
If the “search documentation” keyboard shortcut is also assigned to ^ed, then first remove that binding.
Note that you can also upload the binary over JTAG (program_esp filename.bin 0x10000 verify).
From the VSCode debug side bar (^ed), click on the green arrow at the top and select Launch to connect to the OPNpool device.
Notes
There are only two hardware breakpoints
After hitting a break point, you may still have to select the corresponding task
To determine the image location in flash, OpenOCD uses the address of the first `app` in the partition table. When using OTA updates, this will be the factory image instead of the OTA downloaded application. To work around this, specify esp32 appimage_offset (see docs.espressif.com).
You may only want to debug the 1st core (set ESP32_ONLYCPU 1)
The schematics and layout of the printed circuit board were created using Autodesk EAGLE. If you wish to edit these files, you will need to install the software.
Remember
In Windows, you need to specify the EAGLE_HOME environment variable that points to the your EAGLE directory. Inside EAGLE it is known as $HOME and is used in Control Panel » Options » Directories.
The schematic and board layouts are available in the hardware directory. Custom parts such as the Wemos LOLIN-D32, DC/DC converter, JTAG, RS-485 connector can be found in the hardware/libraries/OPNpool.lbr library.
Go to Options then Directories, and add the absolute path of `OPNpool/hardware/libraries` to the libraries.
ESP Toolchain
These instructions are for Windows, the paths will be slightly different on other operating systems.
Before you start, make sure that Git is installed and that you can call git from the command line.
Install the tools and configure the Espressif IDF extension.
F1 » ISP-IDF: Configure ESP-IDF
Select Advanced mode
ESP-IDF 4.4 in C:/espressif (goes to subdir esp-idf)
GNU Tools in C:/espressif/bin
Choose “Download ESP-IDF Tools”
To speed up compile times, disable Windows Defender’s real-time scanning of C:/espressif. If you get an “You’ll need a new app to open this WindowsDefender link” prompt, then issue Get-AppxPackage Microsoft.SecHealthUI -AllUsers | Reset-AppxPackage from an elevated PowerShell terminal.
If you prefer to use a ESP-IDF beta version or the master branch, refer to VSCode starter for ESP32.
Start humble
From Visual Studio Code, change to an empty folder and copy a simple example project such as blink.
F1 » ESP-IDF: Show ESP-IDF Example Projects
Get-started » Blink
Click on “Create project using example Blink”.
Connect an ESP32 board to your computer. It will show up as a Serial Port device in the Windows device manager.
Select the serial port in Visual Studio Code
F1 » ESP-IDF: Device configuration
Device Target = ESP32
Device Port = COM4
Baud Rate = 115200
Start a compile-flash-monitor cycle
F1 » ESP-IDF: Build, flash and monitor, or use ^ed.
Flash using the “UART”.
In case you encounter warnings or errors, refer to the table below.
Message
Explanation
esptool.py failed with exit code 2
Hints that the baud rate is to high. Try a different/shorter cable.
no such option: -b
Change idf.py monitor line in .vscode\tasks.json so that the command Monitor” is behind the options.
CMake Error: Unable to open check cache file for write.
The directory for the cache file doesn’t exist. Create it by hand.
IDF_PATH environment variable is different from inferred IDF_PATH..
Likely because $IDF_PATH is not set in Windows’ system/user environment. Ignore.
Could NOT find Git (missing: GIT_EXECUTABLE)
Odd because ${env:PATH} does include C:\Program Files\Git\cmd [build system]
Compiling
You should now be able to compile and link e.g. the “factory” application.
Start Visual Studio Code (vscode), in the directory OPNpool\factory.
Connect an ESP32 board to your computer. In Device Manager, the UART (COM) will show up under the Serial Ports. Select this port in Visual Studio Code:
F1 » ESP-IDF: Device configuration
Device Target = ESP32
Device Port = COM6
Baud Rate = 115200
Start a compile-flash-monitor cycle
F1 » ESP-IDF: Build, flash and monitor, or using ^ed.
Flash using the “UART”.
Android Studio
The OPNpool provisioning application is available on the Play Store. The Deploying chapter of this document describes the process in more detail.
If you prefer to build it yourself, download and install Google Android Studio and its SDK. We’d recommend trying to build one of the example projects before you building the OPNpool provisioning app.
Build
Open Android Studio in the android directory and start the build process. Enable Developer mode on you Android device, and enable Wireless Debugging. Your phone should now be listed as a test device. Click the button to run the app on your phone.
Note that the app targets API 30 (Android 11), but builds with API 31. This is so that we can request BLUETOOTH_SCAN and BLUETOOTH_CONNECT permissions for Android 12 devices. For details refer to the Android 12 Behavior Changes document
The design of OPNpool hardware is based around a RS-485 Transceiver and an ESP32 System on a Chip (SoC). The transceiver converts between the RS-485 differential signals and the transmit/receive signals used by the UART in the SoC. The ESP32 is supported by the well documented Espressif IoT Development Framework (ESP-IDF).
Early prototypes
We started by using a single threaded task on a Espressif ESP8266 SoC. This was cheap and offered built-in WiFi, but over time, we migrated to the newer ESP32. Though still very affordable, and it also offers Bluetooth LE and runs FreeRTOS which helps in separating the software components.
1st prototype based on ESP8266, with two RS-232-s and a RS485 transceiver.
2nd prototype based on ESP8266, with RS-232 and RS-485 transceivers.
3rd prototype based on ESP32 with RS-485 transceiver.
4th prototype based on a ESP32 development board with a RS-485 transceiver.
ESP32 SoC
The official name of the ESP32 SoC is “ESP32-D0WDQ6”. It contains two CPUs and peripherals such as WiFi, Bluetooth and Bluetooth LE.
The ESP32-WROOM32 module combines this ESP32 with a clock circuit, 4 MByte flash memory and a trace antenna.
This in turn is mounted on a Wemos LOLIN D32 daughterboard, that provides the reset logic, USB bridge and battery circuit. The main reason for using the LOLIN D32, is that we didn’t want to solder the micro-USB connector ourselves. Looking back, it seems more cost effective as well.
Schematic LOLIN-D32 daughterboard
Schematic
The schematic itself is pretty straightforward. We take the power from the pool controller and convert it to 3.3 V, and a buck converter provides 5 Volts to the battery connector on the ESP32 daughterboard. Using the battery input helps prevent problems when the circuit is also powered through the USB connector
Schematic OPNpool power
The data path is between the RS-485 connector and the ESP32 on the LOLIN D32 daughterboard. There is an optional terminator resistor to prevent reflections on the bus. The JTAG header is for debugging as detailed in the Debugging chapter.
Schematic OPNpool logic
The second revision adds LEDs for RS-485 RX (green) and TX (amber) to simplify setup. It also introduces a push button to preform a factory reset, and increases the bore hole size for the screws.
Bill of Materials
The price comes down to under $40 based on single quantity, excluding shipping and taxes. However, note that some items, such as the PCB, have minimum order quantities.
The pool controller uses a “RS-485” bus to communicate with devices such as the pump and chlorinator. This is an industry standard two-wire bus with differential signaling. Devices are daisy-chained with 120 Ω termination resistors on both ends to limit reflections.
By snooping the traffic on this bus, we can learn about the status of the controller. We can also observe commands that the controller sends to the devices and the subsequent responses that it receives.
Opening the pool controller with a Phillips-head screwdriver
Inside the controller
The easiest place to access the bus is from the inside of the pool controller.
⚠
THIS PROJECT IS OFFERED AS IS. IF YOU USE IT YOU ASSUME ALL RISKS. NO WARRENTIES. PROCEED AT YOUR OWN RISK! Always make sure to at the very least turn off the power while working on your pool equipment. Work carefully, THERE IS ALWAYS A RISK OF BREAKING YOUR POOL EQUIPMENT.
That being said, the RS-485 header can be found on the back of the control board. There are most likely wires already connected that go to devices such as the pump and chlorinator.
Inside of controller
Inside of controller with the connector removed
To minimize electromagnetic interference, use two twisted pairs from something like a CAT-5e cable to connect the A/B pair and the +15V/GND power pair to the interface as shown in the table below. To minimize the risk of shorting the power, you probably don’t want to connect the power signal just yet.
Controller
RS-485
Cat5
Com Port
Name
Name
Idle state
Wire Pair
Green
-DATA
A
negative
pair 1
Yellow
+DATA
B
positive
pair 1
Red
+15 V
n/a
pair 2
Black
GND
n/a
pair 2
RS-485 connection
With our cable connected, we can examine the signals. The shortest pulse length is about 104 µsec. From this, the data rate follows as 1/0.000104 = 9600 baud. The exact flavor of the data bus is shown below.
Name
Value
baud rate
9600
data bits
8
parity
none
stop bits
1
Such a low data rate implies that we don’t have to be very concerned about daisy-chaining the devices. In other words, it is OK to just wiretap the signal on the controller.
When we connect the differential signal to a RS-485 transceiver such as the MAX485, we get regular TTL levels.
Single message
Messages repeat every 2 sec
TTL to CMOS signals
The microcontrollers that we consider use CMOS voltages (3.3 V). For the prototypes, we used an existing breakout board, but replaced the MAX485 with a MAX3485 chip. While we’re at it, we’ll also remove the 10 kΩ CMOS pullup resistors (R1 – R4), but leave the RS-485 bias and bus termination resistors.
Breakout schematic Source: yourduino.com
RS485 with new 3.3V part
Knowing about the interface and baud rate, we can move on to building a system that interprets the proprietary message protocol. Continue reading to learn about the hardware design.
Owning a backyard pool means learning about chemicals, and dealing with covers, vacuuming, brushes and adjusting the thermostats and circuits. A saltwater pool makes things a bit easier, but you still need to keep an good eye on the system.
The OPNpool integrates the functionality of a traditional Pool Controller into the modern smart home. It keeps tabs on the status of the connected controller, pool pump and chlorinator. This provides not only a more convenient solution than physically interacting with the pool equipment, but the ability to create automations that runs the pump for a duration depending on the temperature.
Thanks to the people that reverse engineered RS485-based protocol, OPNpool can listen into the communication between the pool components. The ESP32 shares the resulting state information as JSON/HTTP and can publish it using MQTT. Last but not least, it integrates seamlessly with Home Assistant thanks to MQTT Discovery.
Credentials are provisioned using an Android app. It even supports over-the-air updates when you decide to hack the software!
OPNpool interface
Features:
Visualizes the status of the thermostats, pump, chlorinator, circuits, and schedules.
Allows you adjust the thermostats and toggle circuits
No physical connection to your LAN
Supports over-the-air updates
Easily one-time provisioning from an Android phone
Integrates with MQTT and Home Assistant
Accessible as a webapp
Protected with IP68 waterproof case and connectors
Does not require a power adapter
Open source!
This device was tested with the Pentair SunTouch controller with firmware 2.080, connected to an IntelliFlo pump and IntelliChlor salt water chlorinator.
⚠
This open source and hardware project is intended to comply with the October 2016 exemption to the Digital Millennium Copyright Act allowing “‘good-faith’ testing,” in a controlled environment designed to avoid any harm to individuals or to the public.
We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept”, you consent to the use of ALL the cookies.
This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
Cookie
Duration
Description
cookielawinfo-checkbox-analytics
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional
11 months
The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy
11 months
The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.