improve CI

[edwardhartnett]

I have been applying accumulated experience and knowledge to a refactor of the NCEPLIBS CI systems.

See NCEPLIBS-grib_utils for an example.

Here's a summary of what I had in mind for UFS_UTILS:

• Create a developer build which does address santizer, doxygen build, warning check, code coverage, and Debug build. Turn these settings off in all other builds. Use develop branch of all required NCEPLIBS libraries and the latest version of jasper.
• Create a Linux_versions workflow that checks with different versions of dependencies.
• Create a Linux_settings workflow that tests different build options with the most recent versions of dependencies.
• Create an Intel workflow that uses the Intel compilers.
• Create a MacOS workflow does mac testing, with latest versions.
• Use ubuntu-latest/macos-latest everywhere.
• Use caches everywhere they can be used, including for test data.

For UFS_UTILS, what I propose is to make these changes gradually, over the next week or so, so they do not disrupt any development efforts. I will add workflows. For a time, there will be duplicate testing - as we become confident of the new workflows, we can remove the old ones.

@GeorgeGayno-NOAA any objection to this plan?

[GeorgeGayno-NOAA]

That sounds fine. Can you take the lead? Two of our workflows are currently breaking and I have no idea how to fix them.

[edwardhartnett]

Yes, I will take care of it.

I have just done all the other NCEPLIBS. I was saving UFS_UTILS for last, because it is the most challenging. ;-)

[aerorahul]

@edwardhartnett

Is there a reason for not using spack to build the dependencies?
Kyle Gerheiser, before he left, advised us to use spack to build them as it makes it really easy to.
See the current CI builds for e.g. in UPP, GSI, GSI-utils, WW3.

I am curious what changed and why.

[edwardhartnett]

Nothing has changed. I just haven't tried spack for this yet, but I do plan to.

[arunchawla-NOAA]

Is this still being worked on ? Seems like this should be its own Project board with a number of specific issues as this one issue is too broad

[edwardhartnett]

I will close this issue and add a new one for further work.