Run all tests in the nix-shell; eliminate docker infrastructure
#379
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
spencerkclark
force-pushed
the
nix-debug-mode
branch
3 times, most recently
from
8c75df4
to
ce95f2d
Compare
spencerkclark
force-pushed
the
nix-debug-mode
branch
from
5d09131
to
ea629f5
Compare
spencerkclark
commented
Aug 29, 2023
FV3/gfsphysics/physics/samfdeepcnv.f
Outdated
There was a problem hiding this comment.
These changes are needed to fix debug-mode failures when running with the TKE-EDMF scheme active; they do not change answers. See discussion in #364 (comment) for more context.
spencerkclark
force-pushed
the
nix-debug-mode
branch
4 times, most recently
from
3f048ce
to
cb4185c
Compare
spencerkclark
force-pushed
the
nix-debug-mode
branch
from
cb4185c
to
1e619a4
Compare
spencerkclark
force-pushed
the
nix-debug-mode
branch
from
1e619a4
to
50de16b
Compare
|
After this PR is reviewed I will update the required checks to:
|
brianhenn
reviewed
Aug 31, 2023
brianhenn
reviewed
Aug 31, 2023
brianhenn
reviewed
Aug 31, 2023
tests/pytest/_regtest_outputs/test_regression.test_regression[debug-default.yml-False].out
Show resolved
Hide resolved
brianhenn
reviewed
Aug 31, 2023
There was a problem hiding this comment.
Thanks @spencerkclark things worked well for me. Happy to approve if you like or can wait for another set of eyes.
spencerkclark
force-pushed
the
nix-debug-mode
branch
from
0637993
to
8c2831b
Compare
spencerkclark
added a commit
that referenced
this pull request
Sep 7, 2023
This PR fixes errors in test skipping logic introduced in #379 (see [this CI run](https://app.circleci.com/pipelines/github/ai2cm/fv3gfs-fortran/2682/workflows/e71d0119-0c60-44c7-bf4f-d97f10ccbb0b/jobs/5541)). In that PR these particular tests were configured only to be attempted to be run upon merges to master, because they would be skipped anyway. This PR also now ensures that we at least attempt to run the emulation tests in debug mode in CI when the developer requests to, since it will exercise the skipping logic before merging to master (upon which all pure fortran tests are attempted to be run in both repro and debug mode). [This CI job](https://app.circleci.com/pipelines/github/ai2cm/fv3gfs-fortran/2688/workflows/0c3ca3a2-3b4d-46d6-8cca-a65e1b9a5a0f/jobs/5557) confirms that this PR worked as intended.
This was referenced Sep 14, 2023
spencerkclark
added a commit
that referenced
this pull request
Oct 2, 2023
I noticed this when looking over this file earlier, and it would be good to correct for maximum test coverage. This was introduced in #379, so it has not been present for long.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR refactors the build infrastructure in this repo to eliminate the need for the Docker component. All development and testing is now done in the
nixshell. This should be a quality of life improvement for anyone developing the fortran model, as it no longer requires maintaining checksums in two separate build environments.In so doing it introduces the following changes:
makerules are provided for compiling the model in different modes:build-- build executables inrepro(our production mode) anddebugmode.build_repro-- build only therepromode executable.build_debug-- build only thedebugmode executable.bindirectory, and are tagged with the associated compile mode.check_layout_invariance, is provided to trigger regression tests be run with a 1x2 domain decomposition instead of a 1x1 domain decomposition to check invariance to the domain decomposition layout; this is used for the all the coarse-graining regression tests and replaces the previoustest_run_reproduces_across_layoutstest that would run in the docker image.debug-mode andrepro-mode simulations produce different answers, which is something we noticed in Bump base image to one with Ubuntu version 20.04 #364 when upgrading compiler versions as well, and so require different reference checksums.In working on this PR, we ran the fortran model in
debugmode in more contexts than we had previously, some of which turned up errors, which we currently work around by usingpytest.skip(something we had implicitly already been doing before):Working on this PR also brought my attention to the fact that
pytest'stmpdirfixture does not automatically get cleaned up after each test;pytestversions older than 7.3.0 keep around directories from the last three runs ofpytest, which fill up disk space quickly since running these tests requires creating 10's of run directories, each with their own initial conditions and input files (#380). For the time being I manually clean up these run directories after successful tests.Resolves #340.