Debugging

Coert Vonk

Describes methods for debugging the pool interface. Includes JTAG debugging and post-mortem coredump analysis.

Run time debug

Using `menuconfig`, the user can configure different levels to console logging. The verbosity is specified as None (0), Errors (1), Warnings and errors (2).

  • OPNPOOL_DBGLVL_DATALINK
  • OPNPOOL_DBGLVL_NETWORK
  • OPNPOOL_DBGLVL_POOLSTATE
  • OPNPOOL_DBGLVL_POOLTASK
  • OPNPOOL_DBGLVL_MQTTTASK
  • OPNPOOL_DBGLVL_HASSTASK
  • OPNPOOL_DBGLVL_HTTPD

Post mortem

The device includes support to generate core dumps on unrecoverable errors. This allows post-mortem analysis of the software state at the moment of failure. This way you can find what task, the instruction and what call stack lead to the crash. The core dump is written to the coredump partition in flash.

As we will see in the Interface chapter, the mqtt_task will read this coredump from flash and forward it over MQTT.

GDB over JTAG

The ESP32 supports the Open On-Chip Debugger. Say goodbye to ESP_LOG, and explore the world that was once the exclusive domain of in-circuit emulators. Using special pins on the ESP32, your computer can set break points, inspect variables and single step instructions.

OPNpool with ESP-PROG and computer

ESP32 board and JTAG adapter

The JTAG interface hooks directly to the CPU. That allows it do stop the CPU, set breakpoints and have access to whatever the CPU has access to.

The JTAG header is available on the OPNpool board. This 10 pin header should be connected to a JTAG/USB interface, such as the ESP-PROG. With the interface, make sure to set the JTAG PWR SEL jumper for 3.3V. Also try to keep the connection short (<10 cm).

Another option is to use the ESP-WROVER-KIT development board.
The ESP-WROVER-KIT has an integrated JTAG/USB interface what is available over the first of two USB channels. (The second channel is for the serial port.) If you go this route, make sure to connect the JTAG jumpers on JP2 to prevent Error: libusb_open() failed with LIBUSB_ERROR_NOT_SUPPORTED.

OpenOCD and driver

Under Windows, install the 64-bit FTDI D2xx Driver setup executable;

  1. Using micro-USB cables, connect JTAG adapter and the serial port on the OPNpool board to the computer.
  2. Wait until USB ports are recognized by Windows and drivers are installed. If they do not install automatically, then do so manually using the driver setup from FTDI.
  3. Install and run the Zadig tool (≥ v2.7).
    • Options » List All Devices
    • The list of devices should contain two ESP-PROG specific USB entries with driver name “FTDIBUS (vxxxx)”:
      • Dual RS232-HS (Interface 0), connected to the JTAG port
      • Dual RS232-HS (Interface 1), connected to the UART port
    • Select Dual RS232-HS (Interface 0), and reinstall attached driver to the “WinUSB (v6xxxxx)”, see the picture below.
  4. Windows Device Manager should no longer show the JTAG port, because WinUSB is a user-space driver.
Zadig to configure WinUSB driver.

Compile and debug

Set up the OpenOCD configuration

  1. F1 » Select OpenOCD Board Configuration.
  2. Choose the ESP32 chip (via ESP-PROG). (Unless you use the ESP-WROVER-KIT of course.)
    • Note that WROVER modules use a 1.8 SPI flash because of PSRAM limitations.
    • Others such as WROOM modules use 3.3V SPI flash. This flash voltage is important, because GPIO12 is shared with SPI flash. Selecting the wrong voltage may cause flash uploads to fail.

See if OpenOCD starts

  1. F1 » ESP-IDF: OpenOCD Manager.
  2. Choose Start OpenOCD.

Build the code with symbols and without optimalizations.

  1. F1 » ESP-IDF: Launch GUI configuration tool
  2. Specify the Debug (-Og) compiler optimalization level.
  3. Press ^e d to build, upload and monitor over the serial port.
    • If the “search documentation” keyboard shortcut is also assigned to ^e d, then first remove that binding.
    • Note that you can also upload the binary over JTAG (program_esp filename.bin 0x10000 verify).

From the VSCode debug side bar (^e d), click on the green arrow at the top and select Launch to connect to the OPNpool device.

Notes

  • There are only two hardware breakpoints
  • After hitting a break point, you may still have to select the corresponding task
  • To determine the image location in flash, OpenOCD uses the address of the first `app` in the partition table. When using OTA updates, this will be the factory image instead of the OTA downloaded application. To work around this, specify esp32 appimage_offset (see docs.espressif.com).
  • You may only want to debug the 1st core (set ESP32_ONLYCPU 1)
  • Espressif has more info in Espressif JTAG Debugging

Continue reading to learn about the communication protocol.