Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NetCDF thread safety take two #5095

Merged
merged 67 commits into from
Feb 20, 2023
Merged

NetCDF thread safety take two #5095

merged 67 commits into from
Feb 20, 2023

Commits on Nov 22, 2022

  1. Unpin netcdf4.

    @trexfeathers
    trexfeathers committed Nov 22, 2022
    Configuration menu
    Copy the full SHA
    6caae04 View commit details
    Browse the repository at this point in the history

  2. Temporarily enable GHA on this branch.

    @trexfeathers
    trexfeathers committed Nov 22, 2022
    Configuration menu
    Copy the full SHA
    9dae5d0 View commit details
    Browse the repository at this point in the history

  3. Temporarily enable GHA on this branch.

    @trexfeathers
    trexfeathers committed Nov 22, 2022
    Configuration menu
    Copy the full SHA
    cdcd0e2 View commit details
    Browse the repository at this point in the history

  4. Temporarily enable GHA on this branch.

    @trexfeathers
    trexfeathers committed Nov 22, 2022
    Configuration menu
    Copy the full SHA
    1512708 View commit details
    Browse the repository at this point in the history

  5. Experiment to disable wheel CI on forks.

    @trexfeathers
    trexfeathers committed Nov 22, 2022
    Configuration menu
    Copy the full SHA
    2dc3aff View commit details
    Browse the repository at this point in the history

  6. Disable segfaulting routines.

    @trexfeathers
    trexfeathers committed Nov 22, 2022
    Configuration menu
    Copy the full SHA
    c5ab0e4 View commit details
    Browse the repository at this point in the history

  7. More temporary changes to get CI passing.

    @trexfeathers
    trexfeathers committed Nov 22, 2022
    Configuration menu
    Copy the full SHA
    235e5fa View commit details
    Browse the repository at this point in the history

  8. More temporary changes to get CI passing.

    @trexfeathers
    trexfeathers committed Nov 22, 2022
    Configuration menu
    Copy the full SHA
    efaab93 View commit details
    Browse the repository at this point in the history

  9. Finessed segfault skipping.

    @trexfeathers
    trexfeathers committed Nov 22, 2022
    Configuration menu
    Copy the full SHA
    301dbc6 View commit details
    Browse the repository at this point in the history

  10. Bring in changed from SciTools#5061.

    @pp-mo @trexfeathers
    pp-mo authored and trexfeathers committed Nov 22, 2022
    Configuration menu
    Copy the full SHA
    9ece95c View commit details
    Browse the repository at this point in the history

  11. Re-instate test_load_laea_grid.

    @trexfeathers
    trexfeathers committed Nov 22, 2022
    Configuration menu
    Copy the full SHA
    5103545 View commit details
    Browse the repository at this point in the history

Commits on Nov 24, 2022

  1. Adaptations to get the tests passing.

    @trexfeathers
    trexfeathers committed Nov 24, 2022
    Configuration menu
    Copy the full SHA
    95f337a View commit details
    Browse the repository at this point in the history

  2. Use typing.Mapping instead.

    @trexfeathers
    trexfeathers committed Nov 24, 2022
    Configuration menu
    Copy the full SHA
    fa82db7 View commit details
    Browse the repository at this point in the history

  3. Get doctests passing.

    @trexfeathers
    trexfeathers committed Nov 24, 2022
    Configuration menu
    Copy the full SHA
    b6756d4 View commit details
    Browse the repository at this point in the history

  4. CF only resolve non-url filenames.

    @trexfeathers
    trexfeathers committed Nov 24, 2022
    Configuration menu
    Copy the full SHA
    256936c View commit details
    Browse the repository at this point in the history

Commits on Nov 25, 2022

  1. Confirm thread safety fixes.

    @trexfeathers
    trexfeathers committed Nov 25, 2022
    Configuration menu
    Copy the full SHA
    cf9ba29 View commit details
    Browse the repository at this point in the history

  2. Remove dummy assignment.

    @trexfeathers
    trexfeathers committed Nov 25, 2022
    Configuration menu
    Copy the full SHA
    aed32c8 View commit details
    Browse the repository at this point in the history

  3. Restored plot_nemo What's New entry.

    @trexfeathers
    trexfeathers committed Nov 25, 2022
    Configuration menu
    Copy the full SHA
    835c02d View commit details
    Browse the repository at this point in the history

  4. _add_aux_factories temporarily release global lock.

    @trexfeathers
    trexfeathers committed Nov 25, 2022
    Configuration menu
    Copy the full SHA
    4e2b590 View commit details
    Browse the repository at this point in the history

  5. Remove per-file locking.

    @trexfeathers
    trexfeathers committed Nov 25, 2022
    Configuration menu
    Copy the full SHA
    a8819b4 View commit details
    Browse the repository at this point in the history

  6. Remove remaining test workarounds.

    @trexfeathers
    trexfeathers committed Nov 25, 2022
    Configuration menu
    Copy the full SHA
    8207d6d View commit details
    Browse the repository at this point in the history

  7. Remove remaining comments.

    @trexfeathers
    trexfeathers committed Nov 25, 2022
    Configuration menu
    Copy the full SHA
    c0faa6f View commit details
    Browse the repository at this point in the history

  8. Correct use of CFReader context manager.

    @trexfeathers
    trexfeathers committed Nov 25, 2022
    Configuration menu
    Copy the full SHA
    8010b2f View commit details
    Browse the repository at this point in the history

Commits on Nov 30, 2022

  1. Refactor for easier future maintenance.

    @trexfeathers
    trexfeathers committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    2fb274f View commit details
    Browse the repository at this point in the history

  2. Rename netcdf _thread_safe, add header.

    @trexfeathers
    trexfeathers committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    3aaaa1a View commit details
    Browse the repository at this point in the history

Commits on Dec 1, 2022

  1. Full use of ThreadSafeAggregators.

    @trexfeathers
    trexfeathers committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    02d9a8c View commit details
    Browse the repository at this point in the history

  2. Full use of ThreadSafeAggregators.

    @trexfeathers
    trexfeathers committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    b301120 View commit details
    Browse the repository at this point in the history

  3. Remove remaining imports of NetCDF4.

    @trexfeathers
    trexfeathers committed Dec 1, 2022
    Configuration menu
    Copy the full SHA
    6b7c15c View commit details
    Browse the repository at this point in the history

Commits on Dec 2, 2022

  1. Test to ensure netCDF4 is via _thread_safe module.

    @trexfeathers
    trexfeathers committed Dec 2, 2022
    Configuration menu
    Copy the full SHA
    ab733a7 View commit details
    Browse the repository at this point in the history

  2. More refined netcdf._thread_safe classes.

    @trexfeathers
    trexfeathers committed Dec 2, 2022
    Configuration menu
    Copy the full SHA
    d066d39 View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2022

  1. _thread_safe docstrings.

    @trexfeathers
    trexfeathers committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    cea0e22 View commit details
    Browse the repository at this point in the history

  2. Restore original NetCDF code where possible.

    @trexfeathers
    trexfeathers committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    5ed0ebb View commit details
    Browse the repository at this point in the history

  3. Revert changes to 2.3.rst.

    @trexfeathers
    trexfeathers committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    5b3e609 View commit details
    Browse the repository at this point in the history

  4. Update lockfiles.

    @trexfeathers
    trexfeathers committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    2edb3ce View commit details
    Browse the repository at this point in the history

  5. Merge remote-tracking branch 'upstream/main' into netcdf_segs

    @trexfeathers
    trexfeathers committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    c36c5f5 View commit details
    Browse the repository at this point in the history

Commits on Dec 8, 2022

  1. Additions to _thread_safe.py

    @trexfeathers
    trexfeathers committed Dec 8, 2022
    Configuration menu
    Copy the full SHA
    3059666 View commit details
    Browse the repository at this point in the history

  2. Remove temporary CI shims.

    @trexfeathers
    trexfeathers committed Dec 8, 2022
    Configuration menu
    Copy the full SHA
    97d5891 View commit details
    Browse the repository at this point in the history

Commits on Dec 9, 2022

  1. New locking stategy for NetCDFDataProxy.

    @trexfeathers
    trexfeathers committed Dec 9, 2022
    Configuration menu
    Copy the full SHA
    01be381 View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2022

  1. NetCDFDataProxy simpler use of netCDF4 lock.

    @trexfeathers
    trexfeathers committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    2c972f3 View commit details
    Browse the repository at this point in the history

  2. Update lock files.

    @trexfeathers
    trexfeathers committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    cb49765 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'upstream/main' into netcdf_segs

    @trexfeathers
    trexfeathers committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    140a6c8 View commit details
    Browse the repository at this point in the history

  4. Go back to using a Threading Lock.

    @trexfeathers
    trexfeathers committed Dec 12, 2022
    Configuration menu
    Copy the full SHA
    b2cc5ad View commit details
    Browse the repository at this point in the history

Commits on Jan 9, 2023

  1. Remove superfluous pass commands in test_cf.py.

    @trexfeathers
    trexfeathers committed Jan 9, 2023
    Configuration menu
    Copy the full SHA
    128ebaa View commit details
    Browse the repository at this point in the history

  2. Rename _thread_safe to _thread_safe_nc.

    @trexfeathers
    trexfeathers committed Jan 9, 2023
    Configuration menu
    Copy the full SHA
    8d72578 View commit details
    Browse the repository at this point in the history

  3. Rename thread safe classes to be 'Wrappers'.

    @trexfeathers
    trexfeathers committed Jan 9, 2023
    Configuration menu
    Copy the full SHA
    68588b5 View commit details
    Browse the repository at this point in the history

  4. Better contained getattr and setattr pattern.

    @trexfeathers
    trexfeathers committed Jan 9, 2023
    Configuration menu
    Copy the full SHA
    5d351db View commit details
    Browse the repository at this point in the history

  5. Explicitly name netCDF4 module in _thread_safe_nc docstring.

    @trexfeathers
    trexfeathers committed Jan 9, 2023
    Configuration menu
    Copy the full SHA
    d53d194 View commit details
    Browse the repository at this point in the history

  6. Better docstring for _ThreadSafeWrapper.

    @trexfeathers
    trexfeathers committed Jan 9, 2023
    Configuration menu
    Copy the full SHA
    d9bce6b View commit details
    Browse the repository at this point in the history

  7. Better comment about THREAD_SAFE_FLAG.

    @trexfeathers
    trexfeathers committed Jan 9, 2023
    Configuration menu
    Copy the full SHA
    ed17c83 View commit details
    Browse the repository at this point in the history

  8. list() wrapping within _GLOBAL_NETCDF4_LOCK, to account for generators.

    @trexfeathers
    trexfeathers committed Jan 9, 2023
    Configuration menu
    Copy the full SHA
    9b23b30 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'main' into netcdf_segs

    @trexfeathers
    trexfeathers authored Jan 9, 2023
    Configuration menu
    Copy the full SHA
    8faa2ea View commit details
    Browse the repository at this point in the history

  10. More accurate thread_safe docstrings in netcdf.saver.

    @trexfeathers
    trexfeathers committed Jan 9, 2023
    Configuration menu
    Copy the full SHA
    3a76fc2 View commit details
    Browse the repository at this point in the history

  11. Merge branch 'netcdf_segs' of github.com:trexfeathers/iris into netcd… 

    …f_segs
    @trexfeathers
    trexfeathers committed Jan 9, 2023
    Configuration menu
    Copy the full SHA
    3fb9679 View commit details
    Browse the repository at this point in the history

Commits on Feb 10, 2023

  1. Merge remote-tracking branch 'upstream/main' into netcdf_segs

    @trexfeathers
    trexfeathers committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    af1aa05 View commit details
    Browse the repository at this point in the history

  2. Split netcdf integration tests into multiple modules.

    @trexfeathers
    trexfeathers committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    22f55e3 View commit details
    Browse the repository at this point in the history

Commits on Feb 13, 2023

  1. Tests for non-thread-safe NetCDF behaviour.

    @trexfeathers
    trexfeathers committed Feb 13, 2023
    Configuration menu
    Copy the full SHA
    9d75765 View commit details
    Browse the repository at this point in the history

Commits on Feb 14, 2023

  1. Docstring accuracy.

    @trexfeathers
    trexfeathers committed Feb 14, 2023
    Configuration menu
    Copy the full SHA
    c86881b View commit details
    Browse the repository at this point in the history

  2. Correct use of dask config set (context manager).

    @trexfeathers
    trexfeathers committed Feb 14, 2023
    Configuration menu
    Copy the full SHA
    5a53d36 View commit details
    Browse the repository at this point in the history

  3. Update dependencies.

    @trexfeathers
    trexfeathers committed Feb 14, 2023
    Configuration menu
    Copy the full SHA
    d95f3f8 View commit details
    Browse the repository at this point in the history

Commits on Feb 15, 2023

  1. Review - don't need the first-class import of iris.tests.

    @trexfeathers
    trexfeathers committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    f6eb131 View commit details
    Browse the repository at this point in the history

  2. Better name for the loading test.

    @trexfeathers
    trexfeathers committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    2df6278 View commit details
    Browse the repository at this point in the history

  3. Better selection of data to load.

    @trexfeathers
    trexfeathers committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    2095658 View commit details
    Browse the repository at this point in the history

  4. What's New entry.

    @trexfeathers
    trexfeathers committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    e03659a View commit details
    Browse the repository at this point in the history

  5. Improve tests.

    @trexfeathers
    trexfeathers committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    93ac4c4 View commit details
    Browse the repository at this point in the history

Commits on Feb 20, 2023

  1. Merge remote-tracking branch 'upstream/main' into netcdf_segs

    @trexfeathers
    trexfeathers committed Feb 20, 2023
    Configuration menu
    Copy the full SHA
    3ffe3dc View commit details
    Browse the repository at this point in the history

  2. Update lock files.

    @trexfeathers
    trexfeathers committed Feb 20, 2023
    Configuration menu
    Copy the full SHA
    94da55c View commit details
    Browse the repository at this point in the history

  3. Increase chunking on test_save.

    @trexfeathers
    trexfeathers committed Feb 20, 2023
    Configuration menu
    Copy the full SHA
    a6b893c View commit details
    Browse the repository at this point in the history