Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix tests of process to work with the new FCIS restructure #148

Merged
merged 3 commits into from
Nov 6, 2024

Conversation

blimlim
Copy link
Collaborator

@blimlim blimlim commented Oct 23, 2024

The branch 145/fcis-architecture implements the big restructure of the process and cubewrite functions, as described in #145. The main changes are:

  • IO steps are handled in process(), while all cube modification steps are moved to a new process_cubes() function.
  • The new process_cubes() function absorbs any cube modification steps previously part of cubewrite().
    Roughly process() becomes the "imperative shell", performing all the reading and writing, while process_cubes() forms the "functional core", performing cube modifications over several modular steps.

The new structure caused the existing end to end tests of process() to break. This PR follows on from work in the branch 145/fcis-architecture-broken-test, in getting the tests to work. The changes are:

  • Change the target of the tests to the "functional core" process_cubes()
  • Remove mocks of IO steps
  • Add one mock of fix_time_coord, which uses cube methods including cube.coord_dims(), cube.transpose and iris.util.new_axis which would be harder and fragile to implement in the DummyCube.
  • The existing tests checked that the correct cubes were either dropped or kept by process by checking that the output cubes were the same objects as the input cubes:
    for pc in processed:
            assert pc in cubes
    This doesn't work with the new process_cubes(), as fix_time_coord() and fix_pressure_levels return new cubes rather than modifying in place. These assertions have been swapped to check that the names of the output cubes match what's expected, e.g:
     cube_names = {cube.name() for cube in cubes}
     processed_names = {cube.name() for cube, _, _ in processed}
     assert processed_names == cube_names

The target of this PR is the 145/fcis-architecture branch instead of main. There are several steps before that branch can be merged into main, and hence to try and make the PRs smaller and easier to review, I'll do separate smaller PRs into 145/fcis-architecture, and do a PR into main when all the steps are complete.

Any feedback or suggestions are welcome!

test/test_um2netcdf.py Outdated Show resolved Hide resolved
test/test_um2netcdf.py Outdated Show resolved Hide resolved
test/test_um2netcdf.py Outdated Show resolved Hide resolved
test/test_um2netcdf.py Outdated Show resolved Hide resolved
test/test_um2netcdf.py Outdated Show resolved Hide resolved
Copy link
Contributor

@marc-white marc-white left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be good, pending a clean-up of possibly redundant test fixtures

blimlim and others added 2 commits October 25, 2024 13:17
Copy link
Contributor

@marc-white marc-white left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to go!

Copy link
Collaborator

@atteggiani atteggiani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@atteggiani atteggiani merged commit 2a872f0 into 145/fcis-architecture Nov 6, 2024
@atteggiani
Copy link
Collaborator

Thank you @blimlim!
Merged.

@blimlim blimlim mentioned this pull request Nov 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants