tests/doc: some additions and reorganization of doc page

[krzysztof-cabaj]

Contribution description

During running all tests for few boards I noticed some lack of information in the tests doc page, for example, information how execute all automatic tests for particular board or about cleaning intermediate files. Moreover, current version of the documentation mixes information about running and implementation of tests.

My contribution:

• add section concerning execution of all automate tests for particular boards,
• add section concerning cleaning all intermediate files,
• reorganization of documentation - move to the begin of page all information concerning test execution, later all information concerning implementation,
• change section heading style, from dashed line (first time spotted in this file) to hashes (used in all other doc files).

Testing procedure

make doc
xdg-open doc/doxygen/html/running-and-creating-tests.html

Issues/PRs references

None

[kfessel]

changing the heading style seems ok for me (RiOT-doc just uses both in about equal amount)
reordering makes sense (more likely someone wants to exec (a) test(s))

I think the new cleanup paragraph need some extra words maybe also advertise a less aggressive cleanup strategy.

[kfessel]

distclean might remove a bunch of files that are not test specific.
(e.g. pkg sources, documentation if generated, other applications build)

I think this needs to be written down (warning) (distclean is just very not specific for test)

[krzysztof-cabaj]

Thanks for review! Could you hint me other 'less aggressive cleanup' methods? distclean was the only one which I found when after running tests for three boards my file system was completely full.

[kfessel]

make clean "Cleaning all build products for the current board" If you know know which board you just tested.
This will spare you the pkg-download for the next board you test and the doc rebuild (if you like to have a local doc)

compile_and_test_for_board.py also has a --clean-after flag that might fit your need
or --compile-targets clean if you like to cleanup later

I put me this Makefile into tests/ and examples/ to do test-builds. (just building not testing) (the ignore list are for build chains that i do not have installed, the DONE files generated are for speed up repeated builds) not sure if this helps your Usecase.

DIRS := $(wildcard */)

IGNORE_EXAMPLES := mallinfo \
                  rust-gcoap \
                  rust-hello-world \
                  suit_update \
                  #

IGNORE_TESTS := cpu_avr8_xmega_drivers \
                cpu_efm32_features \
                cpu_efm32_drivers \
                driver_ws281x \
                driver_esp_eth \
                mcuboot \
                pkg_emlearn \
                pkg_nanopb \
                pkg_tensorflow-lite \
                rust_minimal \
                suit_manifest \
                sys_stdio_semihosting \
                ztimer_until \
                #

IGNORE := $(INGNORE) $(IGNORE_EXAMPLES) $(IGNORE_TESTS)

DIRS := $(filter-out $(addsuffix /,$(IGNORE)), $(DIRS))

CLEANLIST := $(addsuffix /bin,$(DIRS))
DIRSDONE := $(addsuffix /bin/DONE,$(DIRS))

all: $(DIRSDONE)
	@echo FINISHED ALL

$(DIRSDONE):
	$(MAKE) -C $(subst /bin/DONE,,$@)
	touch $@

$(DIRS): FORCE
	$(MAKE) -C $@

clean_all: undone
	rm -rf $(CLEANLIST)

undone:  FORCE
	rm $(DIRSDONE)

FORCE:

[krzysztof-cabaj]

Thanks for hints. I will test it and update this section accordingly.

[krzysztof-cabaj]

I update section concerning various methods for cleaning intermediate files. @kfessel could you review my changes?

[kfessel]

reordering 👍
style 👍
reads well

[kfessel]

please squash

[krzysztof-cabaj]

Thanks for review.

All commits squashed.

done

[maribu]

One nitpick inline. I am not insistent on the wording I suggest, feel free to rephrase/extend as you like. Please squash/amend directly.

[kfessel]

This improves the current documentation