Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update QA executable paths tests #30

Merged
merged 4 commits into from
Aug 8, 2024

Conversation

jo-basevi
Copy link
Collaborator

@jo-basevi jo-basevi commented Jul 9, 2024

In an upcoming version of payu, model modules with be loaded using modules use/load in payu config.yaml files, and exe paths for the model/submodel will then be determined by payu using the PATHs added by the module.

  • Removed QA tests for absolute executables paths in config.yaml files, and checking full paths in manifests
  • Instead find the module version in config.yaml, and use that to find the released spack.location file. Then check the executable paths in manifests match an install path in the spack.location, as described in Check executable paths match those deployed for specified model version #18
  • As ACCESS-OM2 has different repository and module for BGC configurations, and the only way to determine that has been checking if "bgc" is in the branch name, the above test has been moved into model specific tests.
  • Added a start of ESM1.5 specific QA test class - under access_esm1p5 marker
  • Added tests for access-om2 configurations - added skeleton files on two configurations to test low/high-res + bgc/non-bgc specific tests. While I was at it, also added a test for bit repro historical for access-om2 using a test resource configuration to check runtime and the checksum file were set as expected.
  • Added test for module in released configurations be /g/data/vk83/modules/access-models( For example for access-om2 rather than /g/data/vk83/apps/spack/0.20/release/modules/linux-rocky8-x86_64)

This requires a new release of payu - as need changes to find exe paths using modules.

This test will fail if using prerelease modules, as there is not a released spack.location file.

Closes #18, Closes #29

Copy link

codecov bot commented Jul 9, 2024

Codecov Report

Attention: Patch coverage is 96.49123% with 2 lines in your changes missing coverage. Please review.

Project coverage is 76.29%. Comparing base (a96160d) to head (a14002a).
Report is 1 commits behind head on main.

Files Patch % Lines
...rc/model_config_tests/test_access_esm1p5_config.py 87.50% 1 Missing ⚠️
src/model_config_tests/test_config.py 97.22% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main      #30       +/-   ##
===========================================
+ Coverage   55.12%   76.29%   +21.16%     
===========================================
  Files          14       15        +1     
  Lines         624      658       +34     
===========================================
+ Hits          344      502      +158     
+ Misses        280      156      -124     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jo-basevi jo-basevi force-pushed the 18-Update-QA-tests-check-exe-paths branch from 0f12363 to c720a08 Compare July 9, 2024 07:02
@jo-basevi jo-basevi marked this pull request as ready for review July 9, 2024 07:04
@CodeGat CodeGat self-requested a review July 9, 2024 07:22
@CodeGat
Copy link
Collaborator

CodeGat commented Jul 10, 2024

Just with regards to the PR comment:

This test will fail if using prerelease modules, as there is not a released spack.location file.

There is also spack.location files for Prereleases as well :)

Should the module use path in released configurations be /g/data/vk83/modules/access-models? For example for access-om2 rather than /g/data/vk83/apps/spack/0.20/release/modules/linux-rocky8-x86_64

I think this would be much cleaner.

Copy link
Collaborator

@CodeGat CodeGat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Just a few comments.

src/model_config_tests/test_config.py Show resolved Hide resolved
src/model_config_tests/test_config.py Show resolved Hide resolved
tests/test_test_access_om2_config.py Show resolved Hide resolved
@jo-basevi jo-basevi force-pushed the 18-Update-QA-tests-check-exe-paths branch from c720a08 to ceb119d Compare August 8, 2024 01:13
- Remove tests checking for absolute exe paths in config.yaml
- Instead test for exe names defined in config.yaml, and that exe paths in manifests is in released spack.location file
- Added another skeleton configuration for dev-025deg_jra55_iaf_bgc to cover the high-res and BGC specific tests
- Added a quick repro test to check runtime was set, and checksum files were as expected for access-om2 model
- Fix formatting of error messages
@jo-basevi jo-basevi force-pushed the 18-Update-QA-tests-check-exe-paths branch from ceb119d to a14002a Compare August 8, 2024 01:20
@jo-basevi jo-basevi merged commit 7053608 into main Aug 8, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants