From 4c80ba1e625dadfba07c1e3c48acd4e7bfee96bf Mon Sep 17 00:00:00 2001 From: Niklas Hauser Date: Wed, 21 Apr 2021 13:50:35 +0200 Subject: [PATCH] [docs] Improve testing Readme --- docs/src/guide/custom-project.md | 2 +- test/README.md | 109 ++++++++++++++----------------- 2 files changed, 51 insertions(+), 60 deletions(-) diff --git a/docs/src/guide/custom-project.md b/docs/src/guide/custom-project.md index 519d9d93c6..493f732649 100644 --- a/docs/src/guide/custom-project.md +++ b/docs/src/guide/custom-project.md @@ -57,7 +57,7 @@ want [to choose one at this point](../../reference/build-systems). Add the ```xml - ../modm/repo.lb + ../../ext/modm/repo.lb diff --git a/test/README.md b/test/README.md index 7115123ca8..f06e37e908 100644 --- a/test/README.md +++ b/test/README.md @@ -1,7 +1,7 @@ # Testing modm modm includes a large number of example as well as an extensive unit test suite -which can be compiled and execute on hosted and embedded targets. We‘ve written +which can be compiled and execute on hosted and embedded targets. We've written our own small test harness to be able to fit the tests onto very small devices like AVRs. As part of our Continuous Integration checks, all examples are compiled and hundreds of unit tests are executed to help us catch regressions. @@ -19,7 +19,7 @@ abilities we have available today. ## Manual Tests -There is plenty of opportunity to customize modm, either using lbuild‘s module +There is plenty of opportunity to customize modm, either using lbuild's module options or simply by using the HAL on your specific hardware. This means the more examples we have of different configurations and use-cases the better we can debug existing problems and prevent new ones. @@ -29,58 +29,70 @@ so that others have an idea of the environment in which it was developed. Test your new code on real hardware with the compiler toolchain we use and make sure to document any caveats you discover. -You can compile all examples locally using our helper script. +You can compile all or parts of the examples locally using our helper script. ```sh - $ cd examples - $ ../tools/scripts/examples_compile.py . +cd examples +../tools/scripts/examples_compile.py . +# Or only a subset of the examples +../tools/scripts/examples_compile.py generic/blinky ``` -!!! tip "Add your example to our CI" - If you created a new folder inside `modm/examples`, you need to add it to - one of the jobs in `.circleci/config.yml`: - ```sh - - run: - name: Examples YOUR TARGET - command: | - (cd examples && ../tools/scripts/examples_compile.py new_folder) - ``` + +## Continuous Integration + +modm uses [GitHub Actions for CI][gha] to compile all the examples and to run +the unit tests on Linux. It also generates a bunch of artifacts that you can +download to check what the CI has been doing. + +If you create a new folder inside `modm/examples`, you need to add it to the +appropriate job in in `.github/workflows/linux.yml`: + +```yml + - name: Examples for new or existing families + if: always() + run: | + (cd examples && ../tools/scripts/examples_compile.py your_example_folder) +``` + +[gha]: https://github.com/modm-io/modm/actions ## Unit Tests Our unit tests are located in the `modm/test` directory and are generated using lbuild as well. They are all submodules of `modm:test` and are all platform -independent. We‘ve written a small Makefile for the most commonly used commands. +independent. We've written a small Makefile for the most commonly used commands. ```sh - $ cd tests - # generates, compiles and executes the unit tests for hosted targets - $ make run-hosted-linux - $ make run-hosted-darwin - # generates and compiles the unit tests for embedded targets - $ make compile-nucleo-f411 - $ make compile-nucleo-f103 - $ make compile-al-avreb-can - # executes the unit tests on the embedded targets - $ make run-nucleo-f411 - $ make run-nucleo-f103 - $ make run-al-avreb-can +cd tests +# generates, compiles and executes the unit tests for hosted targets +make run-hosted-linux +make run-hosted-darwin +# generates and compiles the unit tests for embedded targets +make compile-nucleo-f411 +make compile-nucleo-f103_A +make compile-nucleo-f103_B +make compile-arduino-nano_A # to _H +# executes the unit tests on the embedded targets +make run-nucleo-f411 +make run-nucleo-f103_A +make run-nucleo-f103_B +make run-arduino-nano_A # to _H ``` -!!! info "Monitor the serial output" - The embedded test targets all use the `modm::Board` interface to initialize - the targets and output unit tests results via the default serial connection. +The embedded test targets all use the `modm::Board` interface to initialize the +targets and output unit tests results via the default serial connection. -The unit test library we use is located in `modm/src/unittest` which is the -`modm:unittest` modules. See the existing unit tests for example on how to write -your own. +The unit test library we use is located in `modm/src/unittest` which corresponds +to the `modm:unittest` modules. See the existing unit tests for examples on how +to write your own. -!!! bug "Running unit tests on small targets" - Fitting all unit tests into one executable image is not possible on smaller - AVR and STM32 targets. For these targets only a subset of unit tests must be - selected in the `project.xml` file and multiple images must be executed - manually. We would like to automate this in the future. +Fitting all unit tests into one executable image is not possible on smaller AVR +and STM32 targets. For these targets multiple compile targets generate multiple +images with partial unit tests that must be executed manually. In the future we +would like to execute the unit tests as well as the examples automatically on +the target hardware as well. ## Test all Targets @@ -90,7 +102,6 @@ for ALL targets we support and compile it with a simple GPIO example. This doesn't catch everything, but it does make sure that these modules are at least compilable, which is particularly interesting when writing new peripheral drivers which may have different implementations on other targets. -You may call ```sh $ cd test/all @@ -109,23 +120,3 @@ FAIL 0 Total: 209 $ make run-failed # only run those that failed the last test ``` - -!!! info "Target Compilation Logs" - The individual compilation logs are all saved as artifacts, so that when - something fails, it‘s easy to understand why. - You can access them in [the Artifacts tab of a build][logs]. - - -## Continuous Integration - -modm uses [CircleCI as a Continuous Integration service][circleci]. It compiles -all the examples and executes the unit tests on Linux, then generates and -compiles the full library for a blinky example for all targets that we support. - -In the future we would like to execute the unit tests as well as the examples -automatically on the target hardware as well. However, there isn‘t any -third-party framework to make that easy, so we probably have to write our own. - - -[logs]: https://circleci.com/gh/modm-io/modm/971#artifacts/containers/0 -[circleci]: https://circleci.com/gh/modm-io/workflows/modm/tree/develop \ No newline at end of file