add script to simulate all apps

[JuanSapriza]

I have added an script to build and simulate all apps.

Usage

Make sure the script is executable

sudo chmod u+x test_all.sh

Then source the script and enjoy

. test_all.sh

Considerations

• The execution takes some time
• Currently the script has some fixed parameters:
  • Simulator: Verilator
  • Environment: activates a conda environment named core-v-mini-mcu
  • Linker: On Chip (JTAG) for all simulations
  • Compiler: CLANG and/or GCC (if both are installed, tries both, simulates with GCC)
• The simulation model is built with the following configurations to guarantee all apps can be run (at least to that extent)
  • MEMORY_BANKS=3
  • EXTERNAL-DOMAINS=1
• during the mcu-gen with the above parameters, some files are modified. To aid the user, by the end of the execution it is possible to git stash drop them. This action is optional, but its highly recommended to commit before running the script to avoid having to manually pick the changes to keep.
• Some applications that cannot be run with verilator were added to a blacklist so they are not simulated. Some others were not (as to show the functionality.

Pending features

• Select the simulator
• ⚠️ Fix apps that never finish testing on one simulator
  • example_virtual_flash
  • example_freertos_blinky
  • example_spi_*
• Keep a count of apps that performed a meaningless execution (returned success because the requirements were not met)
• Try different linkers
• Allow modification of certain parameters

[JuanSapriza]

The script is working well for me.
Some interesting tests include:

• Introducing a bug into an app to make it fail the simulation
• Introducing a bug into an app to make it fail the compilation
• Adding/removing apps from the blacklist
• Trying clang
• Trying a different simulator (if that is tested, I can add it to the script)

[davideschiavone]

shall I merge this? do we have an option to specify the LINKER Script and the Target simulator?

[JuanSapriza]

I did not implement any of those to make it as straight forward as possible, so we can start using it.

I would say you (or someone else) tries it as-is. If it works, I would merge and then we can start adding some other features.

[davideschiavone]

i would rather do it now, at least the simulator (questa or verilator) and the linker script.
Also, right now both the compilers are used (clag and gcc), whereas I would leave this as an option again.

Then the CI can use both the options.

[davideschiavone]

Also,

• delete old run verif script
• mention about this script in README (included how to lauch it)

[JuanSapriza]

The things to do (i cannot check the ones in Davide's comment):

• delete old run verif script
• mention about this script in README (included how to launch it)
• Discard the compiler existence check
• Check what happens if the environment is already active
• Let the user choose linker and target simulator

[JuanSapriza]

So far i have tried:

• with and without the environment active
• verilator and questasim (on my pc and eslsrv)
• the different linkers

@davidmallasen is now testing clang

We noticed the important of sourcing for the conda activate command can be run properly (this is included in the documentation)

[JuanSapriza]

David tested clang and questasim and its working for him.

I also added the possibility of changing the timeout.

@davideschiavone Could you read the README modifications and try running it based on that?

[JuanSapriza]

⚠️ Important!

The scripts needs to be SOURCED by calling it as . test_all.sh (not the period . at the beginning)

@davideschiavone could you try it now?