make/testbed-support: fix error with FLASHFILE not defined #12394

aabadie · 2019-10-08T12:26:35Z

Contribution description

This PR is fixing a bug introduced by #12317 when using IOTLAB_NODE. For all nrf boards (and maybe others), FLASHFILE was set based on the programmer used (jlink, openocd, pyocd). Since iotlab is set as default programmer, FLASHFILE is no longer set and this results in an error raised here:

RIOT/Makefile.include

Line 458 in d749e2d

$(error FLASHFILE is not defined for this board: $(FLASHFILE))

This PR is now setting the default FLASHFILE when IOTLAB_NODE is used to the value corresponding to the board (HEXFILE for wsn430, ELFFILE for others).

Note that this PR is also removing the IOTLAB_FLASHFILE variable, since it becomes useless after the proposed fix.

CC'ing @cladmi to also get his opinion on the fix and his possible ideas for improvement.

Testing procedure

Start an experiment on nrf** boards provided by iotlab

$ iotlab-experiment submit -n "" -d 60 -l saclay,nrf51dk,2 -l saclay,nrf52dk,2 -l saclay,nrf52840dk,2

Try to build and flash an example on the boards:

$ make BOARD=nrfxxdk IOTLAB_NODE=auto-ssh -C examples/hello-world

=> on master, you get the following error message:

make: Entering directory '/work/riot/RIOT/examples/hello-world'
/work/riot/RIOT/examples/hello-world/../../Makefile.include:458: *** FLASHFILE is not defined for this board: .  Stop.
make: Leaving directory '/work/riot/RIOT/examples/hello-world

With this PR, it works like a charm for all boards.

Issues/PRs references

Bug introduced by #12317

Makefile.include

miri64 · 2019-10-08T18:13:32Z

Maybe @cladmi should have a look here.

aabadie · 2019-10-11T05:34:22Z

It would be great if @cladmi could have a second look. This is a bug fix, on useful feature (for iot-lab), I think it should be backported once merged.

cladmi · 2019-10-14T12:18:31Z

dist/testbed-support/makefile.iotlab.single.inc.mk

@@ -55,7 +55,7 @@ IOTLAB_USER ?= $(shell cut -f1 -d: $(IOTLAB_AUTH))
 IOTLAB_EXP_ID ?=

 # File to use for flashing
-IOTLAB_FLASHFILE ?= $(ELFFILE)
+FLASHFILE ?= $(ELFFILE)


Why is change required for the fix to work ?
The commit says "useless" without explaining anything or justifying why it is useless.

Not all board have a "protected" programmer configuration that would not set FLASHFILE so it would quite easily use another file. The file used to flash in RIOT can be whichever bin/hex/elf, but on IoT-AB it must be the ELFFILE )except for `wsn430.
I feel that changing this assumes IoT-LAB can flash with whichever file produced. If it is now the case, please clarify this.

git grep 'FLASHFILE ?=' boards/cc2538dk/Makefile.include:FLASHFILE ?= $(BINFILE) boards/chronos/Makefile.include:FLASHFILE ?= $(HEXFILE) boards/common/blxxxpill/Makefile.include: FLASHFILE ?= $(BINFILE) boards/common/msb-430/Makefile.include:FLASHFILE ?= $(HEXFILE) boards/common/msba2/Makefile.include:FLASHFILE ?= $(HEXFILE) boards/common/nrf52/Makefile.include: FLASHFILE ?= $(HEXFILE) boards/common/remote/Makefile.include:FLASHFILE ?= $(BINFILE) boards/common/wsn430/Makefile.include:FLASHFILE ?= $(HEXFILE) boards/f4vi1/Makefile.include:FLASHFILE ?= $(BINFILE) boards/lobaro-lorabox/Makefile.include:FLASHFILE ?= $(BINFILE) boards/mbed_lpc1768/Makefile.include:FLASHFILE ?= $(BINFILE) boards/native/Makefile.include:FLASHFILE ?= $(ELFFILE) boards/nz32-sc151/Makefile.include:FLASHFILE ?= $(BINFILE) boards/opencm904/Makefile.include:FLASHFILE ?= $(BINFILE) boards/openmote-b/Makefile.include: FLASHFILE ?= $(HEXFILE) boards/openmote-cc2538/Makefile.include: FLASHFILE ?= $(BINFILE) boards/pic32-clicker/Makefile.include:FLASHFILE ?= $(HEXFILE) boards/pic32-wifire/Makefile.include: FLASHFILE ?= $(HEXFILE) boards/seeeduino_arch-pro/Makefile.include:FLASHFILE ?= $(HEXFILE) boards/spark-core/Makefile.include:FLASHFILE ?= $(BINFILE) boards/teensy31/Makefile.include:FLASHFILE ?= $(HEXFILE) boards/telosb/Makefile.include:FLASHFILE ?= $(HEXFILE) boards/z1/Makefile.include:FLASHFILE ?= $(HEXFILE) cpu/esp32/Makefile.include:FLASHFILE ?= $(ELFFILE) cpu/esp8266/Makefile.include:FLASHFILE ?= $(ELFFILE) dist/testbed-support/makefile.iotlab.single.inc.mk:IOTLAB_FLASHFILE ?= $(ELFFILE) makefiles/tools/avrdude.inc.mk:FLASHFILE ?= $(HEXFILE) makefiles/tools/bossa.inc.mk:FLASHFILE ?= $(BINFILE) makefiles/tools/dfu.inc.mk:FLASHFILE ?= $(BINFILE) makefiles/tools/edbg.inc.mk:FLASHFILE ?= $(BINFILE) makefiles/tools/jlink.inc.mk:FLASHFILE ?= $(BINFILE) makefiles/tools/openocd.inc.mk:FLASHFILE ?= $(ELFFILE) makefiles/tools/pic32prog.inc.mk:FLASHFILE ?= $(HEXFILE) makefiles/tools/pyocd.inc.mk:FLASHFILE ?= $(HEXFILE) makefiles/tools/uniflash.inc.mk:FLASHFILE ?= $(ELFFILE)

IoT-LAB only supports ELF files now (except for WSN4330) but soon it will also support bin files.

How about using FLASHFILE ?= ELFFILE as default, early (eg before boards Makefile.include are parsed) for all boards except WSN430 ?

If we agree on that, I'll reword the commit message.

I adapted the code because I noticed that it was broken with firefly (iot-lab requires ELFFILE but the build system was already setting BINFILE).

fjmolinas · 2019-10-15T06:39:56Z

I was able to replicate the Issue and test it is fixes by the PR:

Master:

make BOARD=nrf52dk IOTLAB_NODE=auto-ssh -C examples/hello-world
make: Entering directory '/home/francisco/workspace/RIOT/examples/hello-world'
/home/francisco/workspace/RIOT/examples/hello-world/../../Makefile.include:461: *** FLASHFILE is not defined for this board: .  Stop.
make: Leaving directory '/home/francisco/workspace/RIOT/examples/hello-world'

This PR:

make BOARD=nrf52dk IOTLAB_NODE=auto-ssh -C examples/hello-world
make: Entering directory '/home/francisco/workspace/RIOT/examples/hello-world'
Building application "hello-world" for "nrf52dk" with MCU "nrf52".

"make" -C /home/francisco/workspace/RIOT/boards/nrf52dk
"make" -C /home/francisco/workspace/RIOT/boards/common/nrf52xxxdk
"make" -C /home/francisco/workspace/RIOT/core

fjmolinas · 2019-10-15T06:40:32Z

@aabadie please address @cladmi comment, otherwise I also think this should be backported.

fjmolinas · 2019-10-16T09:02:05Z

I noticed this issue also applies to samr21-xpro

fjmolinas · 2019-10-16T09:27:41Z

I tested on the following BOARDS:

BOARDS="iotlab-m3 firefly arduino-zero frdm-kw41z microbit nrf51dk nrf52832-mdk nrf52840-mdk nrf52840dk nrf52dk b-l072z-lrwan1 b-l475e-iot01a samr21-xpro samr30-xpro pba-d-01-kw2x"

master: only b-l072z-lrwan1 b-l475e-iot01a succeed

for board in ${BOARDS}; do IOTLAB_EXP_ID=180250 IOTLAB_NODE=auto-ssh BOARD=${board} make -C examples/hello-world/ flash-only; done
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
iotlab-node --jmespath='keys(@)[0]' --format='int' --id 180250 --list grenoble,m3,98 --update /home/francisco/workspace/RIOT-test/examples/hello-world/bin/iotlab-m3/hello-world.elf | grep 0
0
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
/home/francisco/workspace/RIOT-test/dist/testbed-support/makefile.iotlab.single.inc.mk:110: *** Could not automatically find a node for BOARD=firefly.  Stop.
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
/home/francisco/workspace/RIOT-test/examples/hello-world/../../Makefile.include:461: *** FLASHFILE is not defined for this board: .  Stop.
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
Unknown device, --name=, --path=, or absolute path in /dev/ or /sys expected.
iotlab-node --jmespath='keys(@)[0]' --format='int' --id 180250 --list saclay,frdm-kw41z,1 --update /home/francisco/workspace/RIOT-test/examples/hello-world/bin/frdm-kw41z/hello-world.elf | grep 0
0
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
/home/francisco/workspace/RIOT-test/examples/hello-world/../../Makefile.include:461: *** FLASHFILE is not defined for this board: .  Stop.
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
/home/francisco/workspace/RIOT-test/examples/hello-world/../../Makefile.include:461: *** FLASHFILE is not defined for this board: .  Stop.
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
/home/francisco/workspace/RIOT-test/examples/hello-world/../../Makefile.include:461: *** FLASHFILE is not defined for this board: .  Stop.
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
/home/francisco/workspace/RIOT-test/examples/hello-world/../../Makefile.include:461: *** FLASHFILE is not defined for this board: .  Stop.
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
/home/francisco/workspace/RIOT-test/examples/hello-world/../../Makefile.include:461: *** FLASHFILE is not defined for this board: .  Stop.
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
/home/francisco/workspace/RIOT-test/examples/hello-world/../../Makefile.include:461: *** FLASHFILE is not defined for this board: .  Stop.
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
Unknown device, --name=, --path=, or absolute path in /dev/ or /sys expected.
iotlab-node --jmespath='keys(@)[0]' --format='int' --id 180250 --list saclay,st-lrwan1,10 --update /home/francisco/workspace/RIOT-test/examples/hello-world/bin/b-l072z-lrwan1/hello-world.elf | grep 0
0
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
Unknown device, --name=, --path=, or absolute path in /dev/ or /sys expected.
iotlab-node --jmespath='keys(@)[0]' --format='int' --id 180250 --list saclay,st-iotnode,10 --update /home/francisco/workspace/RIOT-test/examples/hello-world/bin/b-l475e-iot01a/hello-world.elf | grep 0
0
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
/home/francisco/workspace/RIOT-test/examples/hello-world/../../Makefile.include:461: *** FLASHFILE is not defined for this board: .  Stop.
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
/home/francisco/workspace/RIOT-test/examples/hello-world/../../Makefile.include:461: *** FLASHFILE is not defined for this board: .  Stop.
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'

pr: only samr30-xpro, rnf5x-mdk and firefly failed

for board in ${BOARDS}; do IOTLAB_EXP_ID=180254 IOTLAB_NODE=auto-ssh BOARD=${board} make -C examples/hello-world/ flash-only; done
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
iotlab-node --jmespath='keys(@)[0]' --format='int' --id 180254 --list grenoble,m3,98 --update /home/francisco/workspace/RIOT-test/examples/hello-world/bin/iotlab-m3/hello-world.elf | grep 0
0
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
/home/francisco/workspace/RIOT-test/dist/testbed-support/makefile.iotlab.single.inc.mk:107: *** Could not automatically find a node for BOARD=firefly.  Stop.
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
iotlab-node --jmespath='keys(@)[0]' --format='int' --id 180254 --list saclay,arduino-zero,1 --update /home/francisco/workspace/RIOT-test/examples/hello-world/bin/arduino-zero/hello-world.elf | grep 0
0
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
Unknown device, --name=, --path=, or absolute path in /dev/ or /sys expected.
iotlab-node --jmespath='keys(@)[0]' --format='int' --id 180254 --list saclay,frdm-kw41z,1 --update /home/francisco/workspace/RIOT-test/examples/hello-world/bin/frdm-kw41z/hello-world.elf | grep 0
0
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
iotlab-node --jmespath='keys(@)[0]' --format='int' --id 180254 --list saclay,microbit,1 --update /home/francisco/workspace/RIOT-test/examples/hello-world/bin/microbit/hello-world.elf | grep 0
0
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
/home/francisco/workspace/RIOT-test/dist/testbed-support/makefile.iotlab.single.inc.mk:107: *** Could not automatically find a node for BOARD=nrf51dk.  Stop.
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
/home/francisco/workspace/RIOT-test/dist/testbed-support/makefile.iotlab.single.inc.mk:90: *** Could not find 'archi' for nrf52832-mdk, update mapping in /home/francisco/workspace/RIOT-test/dist/testbed-support/makefile.iotlab.single.inc.mk.  Stop.
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
/home/francisco/workspace/RIOT-test/dist/testbed-support/makefile.iotlab.single.inc.mk:90: *** Could not find 'archi' for nrf52840-mdk, update mapping in /home/francisco/workspace/RIOT-test/dist/testbed-support/makefile.iotlab.single.inc.mk.  Stop.
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
iotlab-node --jmespath='keys(@)[0]' --format='int' --id 180254 --list saclay,nrf52840dk,10 --update /home/francisco/workspace/RIOT-test/examples/hello-world/bin/nrf52840dk/hello-world.elf | grep 0
0
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
iotlab-node --jmespath='keys(@)[0]' --format='int' --id 180254 --list saclay,nrf52dk,10 --update /home/francisco/workspace/RIOT-test/examples/hello-world/bin/nrf52dk/hello-world.elf | grep 0
0
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
Unknown device, --name=, --path=, or absolute path in /dev/ or /sys expected.
iotlab-node --jmespath='keys(@)[0]' --format='int' --id 180254 --list saclay,st-lrwan1,10 --update /home/francisco/workspace/RIOT-test/examples/hello-world/bin/b-l072z-lrwan1/hello-world.elf | grep 0
0
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
Unknown device, --name=, --path=, or absolute path in /dev/ or /sys expected.
iotlab-node --jmespath='keys(@)[0]' --format='int' --id 180254 --list saclay,st-iotnode,10 --update /home/francisco/workspace/RIOT-test/examples/hello-world/bin/b-l475e-iot01a/hello-world.elf | grep 0
0
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
iotlab-node --jmespath='keys(@)[0]' --format='int' --id 180254 --list saclay,samr21,12 --update /home/francisco/workspace/RIOT-test/examples/hello-world/bin/samr21-xpro/hello-world.elf | grep 0
0
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
/home/francisco/workspace/RIOT-test/dist/testbed-support/makefile.iotlab.single.inc.mk:107: *** Could not automatically find a node for BOARD=samr30-xpro.  Stop.
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
make: Entering directory '/home/francisco/workspace/RIOT-test/examples/hello-world'
Unknown device, --name=, --path=, or absolute path in /dev/ or /sys expected.
iotlab-node --jmespath='keys(@)[0]' --format='int' --id 180254 --list saclay,phynode,1 --update /home/francisco/workspace/RIOT-test/examples/hello-world/bin/pba-d-01-kw2x/hello-world.elf | grep 0
0
make: Leaving directory '/home/francisco/workspace/RIOT-test/examples/hello-world'

The following boards rf52832-mdk nrf52840-mdk are not mapped in makefile.iotlab.single.inc.mk, @aabadie is there a reson for this?

firefly was not found for some reason.. although I did reserver the board.

samr30-xpro state is tagged as unkown in my expriment.

aabadie · 2019-10-16T09:32:24Z

rf52832-mdk nrf52840-mdk

#12395 adds support for the 52840-mdk. I can add a commit there for the 52832-mdk (I thought it was already supported...).

samr30-xpro are broken on Saclay... firefly I don't know, maybe some problems on the Lille site.

aabadie · 2019-10-16T09:34:06Z

I can add a commit there for the 52832-mdk (I thought it was already supported...).

Done.

And thanks for testing by the way !

fjmolinas · 2019-10-16T09:37:43Z

@aabadie I think this is good, please squash.

This way FLASHFILE is either already defined by the board and, if not, the default one for iotlab can be used.

aabadie · 2019-10-16T09:45:26Z

Done!

fjmolinas · 2019-10-16T09:50:00Z

@cladmi I think your comments where addressed, are you ok with this?

fjmolinas

@cladmi I think your comments where addressed, are you ok with this?

From what I can see the comments have been addressed, I have tested the changed. ACK!

@aabadie can you provide the back-port PR?

aabadie added Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors) Area: build system Area: Build system CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Oct 8, 2019

aabadie requested a review from fjmolinas October 8, 2019 12:26

aabadie mentioned this pull request Oct 8, 2019

dist/testbed-support: add nrf52{840,832}mdk in IOTLAB_NODE targets #12395

Merged

miri64 reviewed Oct 8, 2019

View reviewed changes

Makefile.include Show resolved Hide resolved

aabadie force-pushed the pr/make/iotlab_support_fix branch 2 times, most recently from e6ad671 to 7549b1c Compare October 9, 2019 13:05

cladmi reviewed Oct 14, 2019

View reviewed changes

aabadie force-pushed the pr/make/iotlab_support_fix branch from 7549b1c to a506a6d Compare October 15, 2019 08:47

aabadie added 2 commits October 16, 2019 11:44

Makefile.include: include iotlab.single.inc.mk before checking FLASHFILE

41a2249

This way FLASHFILE is either already defined by the board and, if not, the default one for iotlab can be used.

dist/testbed-support: boards: remove useless IOTLAB_FLASHFILE

ffc4c84

aabadie force-pushed the pr/make/iotlab_support_fix branch from a506a6d to ffc4c84 Compare October 16, 2019 09:45

fjmolinas approved these changes Oct 17, 2019

View reviewed changes

fjmolinas merged commit 0177950 into RIOT-OS:master Oct 17, 2019

aabadie deleted the pr/make/iotlab_support_fix branch October 18, 2019 06:48

aabadie mentioned this pull request Oct 18, 2019

make/testbed-support: fix error with FLASHFILE not defined [backport 2019.10] #12493

Merged

kb2ma added this to the Release 2019.10 milestone Oct 27, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

make/testbed-support: fix error with FLASHFILE not defined #12394

make/testbed-support: fix error with FLASHFILE not defined #12394

aabadie commented Oct 8, 2019

miri64 commented Oct 8, 2019

aabadie commented Oct 11, 2019

cladmi Oct 14, 2019

aabadie Oct 15, 2019

aabadie Oct 15, 2019

fjmolinas commented Oct 15, 2019

fjmolinas commented Oct 15, 2019

fjmolinas commented Oct 16, 2019

fjmolinas commented Oct 16, 2019

aabadie commented Oct 16, 2019

aabadie commented Oct 16, 2019

fjmolinas commented Oct 16, 2019

aabadie commented Oct 16, 2019

fjmolinas commented Oct 16, 2019

fjmolinas left a comment

make/testbed-support: fix error with FLASHFILE not defined #12394

make/testbed-support: fix error with FLASHFILE not defined #12394

Conversation

aabadie commented Oct 8, 2019

Contribution description

Testing procedure

Issues/PRs references

miri64 commented Oct 8, 2019

aabadie commented Oct 11, 2019

cladmi Oct 14, 2019

Choose a reason for hiding this comment

aabadie Oct 15, 2019

Choose a reason for hiding this comment

aabadie Oct 15, 2019

Choose a reason for hiding this comment

fjmolinas commented Oct 15, 2019

fjmolinas commented Oct 15, 2019

fjmolinas commented Oct 16, 2019

fjmolinas commented Oct 16, 2019

aabadie commented Oct 16, 2019

aabadie commented Oct 16, 2019

fjmolinas commented Oct 16, 2019

aabadie commented Oct 16, 2019

fjmolinas commented Oct 16, 2019

fjmolinas left a comment

Choose a reason for hiding this comment