Deploying

Rolling out a device to multiple sites introduces challenges. The WiFi credentials can no longer be part of the source code. Applying firmware updates or accessing crash information over USB becomes cumbersome. Even simply finding the IP address of the device on the LAN may be challenging.

To address such challenges we use a three tiered approach, in which the bootloader image decides what image to boot.

1. As usual, the bootloader image does some minimum initializations. If it finds a valid ota image, it passes control over to that image. If not, it starts the factory image.
2. The factory image takes care of provisioning WiFi and MQTT credentials with the help of a phone app. These credentials are stored in the nvs partition. It then downloads the ota image.
3. We refer to the ota image as the interface, as it provides the core of the functionality of the OPNpool device.

To facilitate this the flash memory is partitioned as shown below.

The partition table is shown in the table below.

The following sections describes the process in further detail. A whole chapter is dedicated to the interface image, which provides the core functionality of the OPNpool device.

Bootloader image

A few more words on the bootloader:

• If otadata partition is erased, it starts the factory image (assuming it is present).
• After the first OTA update, the ota_data partition is updated to specify which OTA app slot partition should be booted next.
• If the image works fine, it marks itself as ESP_OTA_IMG_VALID in the ota_data. Otherwise, it marks itself ESP_OTA_IMG_INVALID. This mechanism support image rollback to keep the device working after the update. It automatically rolls back to the previous version, if the image doesn’t pass its self test.

Factory image

The factory image, handles the provisioning and triggers an over-the-air (OTA) download of the interface image. It then reboots, the bootloader will start the interface image.

These steps are described in detail in the following section.

Provisioning WiFi credentials

For testing purposes, WIFI_CONNECT_SSID, WIFI_CONNECT_PASSWORD and OPNPOOL_MQTT_URL can also be provisioned using Kconfig. If empty, the device will use credentials from flash memory.

A phone is used to connect to the factory app using Bluetooth Low Energy (BLE). The factory image will advertise itself to the phone app. Using this phone the user specifies the WiFi and MQTT credentials. During the process the phone remains in contact with the ESP while the WiFi is set up and connects to an access point. The WiFi and MQTT credentials are stored in the nvs partition of flash memory.

You can find the source code of this app in the android directory. If you have an iOS phone, or you have problems running the Android app, you can extend esp_prov.py to include mqtt_url similar to what is shown here“.

The code base uses the git submodule ESP32_factory-ble-prov.

The figure and video below give an impression of the provisioning process

I (907) factory: Starting BLE provisioning
I (1338) ble_prov: advertising as "POOL_CC4504"
I (99768) ble_prov_handler: Received WiFi credentials:
        ssid Guest Barn
        password xxxxxxxxxx
I (100098) ble_prov_handler: WiFi Credentials Applied
I (100288) ble_prov_handler: Connecting ..
I (104328) factory: IP addr 10.1.1.118
I (104328) ble_prov: STA Got IP
I (104328) ota_task: Checking for OTA update (https://coertvonk.com/cvonk/pool/interface.bin)
I (104338) ota_task: Running from part "factory" (0x00010000)
I (104548) ota_task: Writing part ota_0 at offset 0x160000
I (104548) ota_task: Firmware on server: interface.f6d8367-dirty (Mar  2 2022 10:26:42)
I (104548) ota_task: Firmware running:   factory.6bc4c65-dirty (Mar  2 2022 14:52:42)
W (104558) ota_task: Downloading OTA update ..
I (105848) ble_prov_handler: Connected state
I (109668) ota_task: Wrote 5% of 1280 kB
:

Note to self: needs to check MQTT connection status and OTA download progress, awaiting answer to “Provisioning with custom-data after device establishes Wi-Fi connection“.

Connecting to the WiFi network

To establish the WiFi connection, the code base uses the git submodule ESP32_wifi-connect. This component makes this API available as a reusable component. It connects to a WiFi Access Point, and automatically reconnects when the connection drops.

Download the Interface image

The ESP-IDF API provides a mechanism that allows a device to update itself based on data received while the normal firmware is running. Our code base uses the git submodule ESP32_ota-update-task that makes the API available as a component.

The location of the factory image is specified by OTA_UPDATE_FIRMWARE_URL in Kconfig.

The code checks for an OTA update on a network server. If the image is different, it will download it. Upon completion, the device resets to activate the downloaded code. Note that we use the term “update” loosely, because it can also be used to downgrade the firmware.

To determine if the currently running code is different as the code on the server, it compares the project name, version, date and time. Note that these are not always updated by the SDK. The best way to make sure they are updated is by committing your code to Git and building the project from scratch (by removing the build directory).

During debugging, you may want to flash the image image directly as part of the “IDF-SDK: Build, flash and start monitor your device” cycle. To prevent that image from being overwritten by an OTA update, the update mechanism is disabled when the interface image is loaded in the factory partition.

An example of the update process is shown in the figure below.

I (3222) ota_task: Checking for OTA update (https://coertvonk.com/cvonk/pool/interface.bin)
    I (3232) ota_task: Running from part "factory" (0x00010000)
    I (3412) ota_task: Writing part ota_0 at offset 0x160000
I (3422) ota_task: Firmware on server: interface.f6d8367-dirty (Mar  2 2022 10:26:42)
I (3422) ota_task: Firmware running:   interface.6bc4c65 (Mar  2 2022 10:29:53)
W (3432) ota_task: Downloading OTA update ..
I (8142) ota_task: Wrote 5% of 1280 kB
:
I (23602) ota_task: Connection closed
I (24322) ota_task: Prepare to restart system!

More details of this component can be found at Github.