Debugging

Tips and tricks on how to debug code related to the Toit framework.

Running Specific Tests

One can run individual test using the ctest command:

cd build/host
ctest --verbose -R test_name

ctest does some matching, so the test name doesn't need to be complete.

The --verbose flag also gives the full command line that was used to run the command. Note that some tests require to be run from specific locations. Generally, we only have two important locations, though: the root of the checkout, or the directory the test is in.

Tests are discovered by CMake with a glob. New tests are only discovered if cmake is rerun. You can force a cmake regeneration with make rebuild-cmake (but a lot of other make targets will automatically do that too).

GDB

Compile the Toit executable with -O0 and in debug mode:

rm -rf build
LOCAL_CXXFLAGS="-O0" make BUILD_TYPE=Debug

By default the compiler is run with fork (to release its memory when it's done). You can avoid that by running with the -Xno_fork flag:

gdb --args build/host/sdk/bin/toit.run -Xno_fork hello.toit

Valgrind

Valgrind is great at detecting some of the typical C++ mistakes (uninitialized memory and such). It can also detect memory leaks.

It is recommended to compile the Toit executables in debug mode (see the GDB section).

Since the compiler does not release memory (it is a short-running process and relies on fork to clean everything up), that can introduce a lot of false positives. When looking for memory leaks, it is recommend to start from snapshots so that the compiler isn't invoked.

build/host/sdk/bin/toit.compiler -w out.snapshot hello.toit
valgrind build/host/sdk/bin/toit.run out.snapshot

JTAG

Setup

We will use an FT2232H (or FT2232HL, where "L" stands for "lead free") together with OpenOCD. The FT232H should also work. It's the same chip as the FT2232H but has only one channel.

The FT2232H boards are relatively cheap (~$15 on aliexpress), and the FT232H boards are usually below $10.

Connect:

• AD0 - GPIO13 (TCK)
• AD1 - GPIO12 (TD1)
• AD2 - GPIO15 (TD2)
• AD3 - GPIO14 (TMS)
• GND - GND

On my board the pins are as follows:

• AD0 - orange
• AD1 - yellow
• AD2 - green
• AD3 - blue
• GND - black

Paste the following content into a ft2232hl.cfg (or ft232h.cfg):

# See https://www.allaboutcircuits.com/technical-articles/getting-started-with-openocd-using-ft2232h-adapter-for-swd-debugging/
# for an explanation of these configuration entries.
interface ftdi
# Update these numbers with the ones given by lsusb.
ftdi_vid_pid 0x0403 0x6010
ftdi_channel 0
# just TCK TDI TDO TMS, no reset
# This line and comment comes from https://www.esp32.com/viewtopic.php?t=15952#p60800
ftdi_layout_init 0x0038 0x003b
# ESP only supports jtag.

transport select jtag
adapter_khz 200

Start openocd:

# The 'target/esp32.cfg' is part of openocd and on its "configuration-path".
openocd -f ft2232hl.cfg -f target/esp32.cfg

If the wiring is correct you should see:

...
Info : JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : JTAG tap: esp32.cpu1 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
...

If not, check your wiring.

sdkconfig

If you are debugging crashes, consider the following sdkconfigs:

CONFIG_ESP_SYSTEM_PANIC_GDBSTUB=y
CONFIG_ESP_DEBUG_OCDAWARE=y

• CONFIG_ESP_SYSTEM_PANIC_GDBSTUB starts a GDB stub when a panic is encountered allowing you to debug the exact location where things went wrong. Disconnect GDB first, and then, when the message "Setting breakpoint at ..." has been printed, connect with your GDB.
• CONFIG_ESP_DEBUG_OCDAWARE stops the ESP from rebooting when it encounters a panic.

Running gdb

Run the xtensa-esp32-elf-gdb inside the 'toit' checkout so that the sources are mapped.

Usually you want to start with the following commands:

set remote hardware-watchpoint-limit 2
file build/esp32/toit.elf
target extended-remote localhost:3333

Useful commands:

mon reset halt  # Reset the chip and keep the CPUS halted.
maintenance flush register-cache  # (After 'mon reset halt').

thb app_main  # insert a temporary hardware breakpoint at 'app_main'

The esp-idf has the following GDB commands in its source:

# Add Python GDB extensions
python
import sys
sys.path = {sys_path}
import freertos_gdb
end
...
set confirm off
  add-symbol-file {boot_elf}
set confirm on
...
# Connect to the default openocd-esp port and break on app_main()
target remote :3333
monitor reset halt
flushregs
thbreak app_main
continue