migrate all boards upload tool from bossac 1.8 to 1.9 and auto-generate boards.txt

[hathach]

• mostly for fixing the issue Installation of "adafruit:samd" blocked by dependency on bossa(c) 1.8 (not available for the current OS) arduino-board-index#95 since Arduino provide pre-builtin binaries for aarch64 from version 1.9
• I have go through bossac 1.8 --> 1.9 commit log, https://github.com/shumatech/BOSSA/commits/master . The changes does not seem to include any breaking changes, 1.9 is fully compatible with 1.8 in term of options/arguments.
• Furthermore, 1.9 include more fixes and enhance support especially for SAMD21/D51
  • shumatech/BOSSA@f7a65bb
  • shumatech/BOSSA@df65c45
  • shumatech/BOSSA@deecb80

Also do a bit of testing with actual hardware feather m0, feather m4, proxlight trinkey all seem to work well.

[dhalbert]

bossac19 is very dangerous if you forget the --offset argument. The default is 0x000, and so it will wipe the bootloader. and the fuses The default was incompatibly changed from 1.8 to 1.9.

https://learn.adafruit.com/welcome-to-circuitpython/non-uf2-installation#run-the-bossac-command-3000302-16

[dhalbert]

• erase option wipes SAMD21 protected bootloader, fuses shumatech/BOSSA#99
• "Removed knowledge of the bootloader size from BOSSA" commit is dangerously compatibility-breaking shumatech/BOSSA#79
  for example

[hathach]

thanks @dhalbert luckily we did specific --offset in command line option https://github.com/adafruit/ArduinoCore-samd/blob/bossac-1.9/platform.txt#L161 But it is indeed dangerous, I saw that commit in bossa log, but didn't aware of what it means.

Sum up: we are still good, since offset is specified for all boards.

[dhalbert]

We could fork bossac and protect against this. I am mostly worried about people using the installed bossac manually and then bricking their boards.

[hathach]

We could fork bossac and protect against this. I am mostly worried about people using the installed bossac manually and then bricking their boards.

Arduino user won't probably using bossac manually, I have never run bossac manually before as well. However, in case of bricking, is it possible to erase and re-flash the bootloader ?

Update: just realized that bossac is just a bootloader client 🤦, user would need an jlink to reflash bootloader

[ladyada]

isnt the bootloader fuse protected against writing to 0x0 by accident?

[dhalbert]

If BOOTPROT is set, then the bootloader will be protected. But if the offset is zero (the default), it appears to use the ERASE ALL command, and sets the fuses to all ones: shumatech/BOSSA#99 (comment).

[ladyada]

huh and BOOTPROT doesnt listen to that?

[dhalbert]

huh and BOOTPROT doesnt listen to that?

It doesn't overwrite the bootloader, but it does overwrite the fuses with all ones (apparently). I can try this myself.

[hathach]

I have done more reading on this, bossac 1.9 only erase-all if -e option is passed to command line. We don't actually use -e option in 1.8 and 1.9 upload pattern at all, only -w for sector erase. So I guess it is safe enough with offset and no -e

ArduinoCore-samd/platform.txt

Line 161 in 6bc12c9

tools.bossac19.upload.pattern="{path}/{cmd}" {upload.verbose} --port={serial.port.file} -U -i --offset={upload.offset} -w -v "{build.path}/{build.project_name}.bin" -R

IMHO, we don't really need to worry about user manual execution of the command. Actually bossac 1.9 will probably be downloaded in user arduino15 already since it is an dependency tool required by Arduino mbed core. Mine have all 1.7, 1.8 and 1.9

[hathach]

@dhalbert @ladyada in order to minimize the mistake in manual addition of a board e.g incorrect offset for samd21/samd51. I have added the makeboard.py script

• https://github.com/adafruit/ArduinoCore-samd/blob/bossac-1.9/extras/makeboards.py

The generated boards.txt does fix some existing typo. The diff may look large but it is mostly macro arrangement in the extra_flags since we don't do in consistent order currently. Please review and let me know if changes makes sense to you.

[hathach]

@ladyada I made some highlight for the changes in generated boards.txt , please check out if you have time and/or if it makes sense to you. Let me know if you want to make any changes to the script/generated boards.txt

[dhalbert]

It doesn't overwrite the bootloader, but it does overwrite the fuses with all ones (apparently). I can try this myself.

I used bossac v1.9.1 to write a .bin at offset zero, with -e, and the fuses did not get overwritten, with and without BOOTPROT set properly. There is fuse repair code in the bootloader, but I changed the fuses slightly to notice whether it was triggered or not, and it wasn't. So never mind about this. I still wish bossac hadn't made this dangerous change, but it does appear we can live with it.

[ladyada]

i like we have makeboards.py - it will be very very helpful given the large qty of PCBs!

[hathach]

thank you @dhalbert for helpful information and hand-on testing. I am glad we got everything covered safely now.

[dansteingart]

Thanks for this thread: is there a new boards .json I can point to? Everything still seems to reference arduino:bossac@1.8.0-48-gb176eee. TIA.

[hathach]

@dansteingart we reverted the bossac back to 1.8 due to 1.9 compatible issue with macos #323 . And we won't upgrade it back until the issue is fixed by bossac. If you are using aarch64, you could try to install 1.7.8 instead. I am working on manual way to install bsp for aarch64 user (but in middle of other works just now).

[dansteingart]

Thanks: apologies for the dumb question but how do I override the bossac chain? I’ve replaced the library in the package.json install, and I can compile correctly, but something is still looking for 1.8

Error during Upload: Failed uploading: cannot execute upload tool: fork/exec {runtime.tools.bossac-1.8.0-48-gb176eee.path}/bossac: no such file or directory

[hathach]

copy your bossac binary to following path of arduino15, create middle folder if needed:

arduino15/packages/arduino/tools/bossac/1.8.0-48-gb176eee/

[hathach]

if adafruit/arduino-board-index#98 is approved then aarch64 will just work with 1.7.10 without any hassle.