Add miniapps as tests

[msimberg]

Adds the algorithms miniapps as tests. Does not add miniapp_laset as it's not really testing anything too useful, and it messes with the check-threads setup (which checks if OS threads are being spawned that shouldn't be spawned). The miniapps are run with 6 ranks, --grid-rows 3 --grid-cols 2, and otherwise default options.

This splits DLAF_addTest into two separate functions: one which adds a test given a target (DLAF_addTargetTest) which is used to add the miniapps since they already have CMake targets, and DLAF_addTest which creates an executable and then calls DLAF_addTargetTest. The behaviour of DLAF_addTest remains unchanged.

This also adds a CATEGORY parameter that can be added to tests as a label. This is added alongside the RANK_* labels. The CATEGORY defaults to UNIT for "unit tests", and I set it to MINIAPP for the miniapps. I'm fine with making the choice explicit and/or renaming/adding the categories. I then use the RANK and CATEGORY labels to generate jobs the same way RANK is used currently on master. Combinations that have no tests are not added as CI jobs.

Finally, many miniapps were hanging on the CUDA configurations, where they run with only one worker thread and one MPI thread. I've changed the waitLocalTiles calls to pika::wait calls before MPI_Barrier is called to make sure that really nothing is running anymore and deadlocks are avoided. I've only changed them to pika::wait after the algorithms are called, but for consistency it might make sense to use pika::wait anywhere before MPI_Barrier is called?

[msimberg]

cscs-ci run

[msimberg]

cscs-ci run

[msimberg]

cscs-ci run

[msimberg]

cscs-ci run

[msimberg]

cscs-ci run

[msimberg]

cscs-ci run

[msimberg]

cscs-ci run

[msimberg]

cscs-ci run

[msimberg]

cscs-ci run

[msimberg]

cscs-ci run

[msimberg]

cscs-ci run

[msimberg]

cscs-ci run

[rasolca]

Not happy about spreading pika::waits everywhere in the miniapps.
I'd prefer another solution.
Would scheduling and sync_waiting an Ibarrier work?

[msimberg]

cscs-ci run

[msimberg]

cscs-ci run

[msimberg]

cscs-ci run

[msimberg]

cscs-ci run

[msimberg]

cscs-ci run

[albestro]

I was experiencing hangs on miniapp_eigensolver just after the new transform_mpi #1125. I don't think that's the direct cause of them, but it seems like it exposed a problem.

I was able to replicate the hang on both daint (with 8 nodes), and on todi (with 4 and 8 nodes). I did some basic investigation, and miniapp check was the discriminant between hang (enabled) or no hang (disabled). Actually in one of my tests I tried disabling just hermitian_multiplication communications and it was enough to not make it hang anymore.

As suggested by @msimberg talking about this, I tried adding pika::wait() in the dlaf::finalize() and actually that resolved hangs in my problem reproducing configurations. Moreover, I opted for giving a try to wait_all_communicators changes (alone) from this PR, and also them are enough to fix hangs in my test-case.

[msimberg]

cscs-ci run