Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix tests #545

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Fix tests #545

wants to merge 2 commits into from

Conversation

aulemahal
Copy link
Contributor

@aulemahal aulemahal commented Nov 7, 2022

Change Summary

Pin netCDF4 to a version lower than 1.6. I've lost track of the issues talking about the problems of netCDF4 1.6.1, but downgrading solved the errors in all CI errors, but one. I'm not sure, but this could be the culprit issue : Unidata/netcdf4-python#1192

The test_source.py::test_open_dataset_kerchunk test is failing in a weird way. It seems that at some point, fsspec fails with a ConnectionTimeoutError, however, this error is not raised. Instead somehow replaces an object and the fsspec later tries to access an item of that object, this time raising TypeError: 'ConnectTimeoutError' object is not subscriptable.

The noaa-nwm-test-reference.json file made reference to the S3 bucket noaa-nwm-retro-v2.0-pds and folder full_physics. I have not found this bucket by googling it. When trying to access the http endpoint of s3 directly, it failed. However, I did find this:
https://registry.opendata.aws/nwm-archive/ . Was the old bucket simply renamed and reorganized? This was done in #480.
@andersy005 any ideas?

I replaced all reference of the old urls with a new one that imitates the one that works for me in the http browser, but it still fails.

Checklist

  • Unit tests for the changes exist
  • Tests pass on CI
  • Documentation reflects the changes where applicable

@aulemahal
Copy link
Contributor Author

Oh! I didn't realize the error was different with the changed s3 paths. Now it's a "NoCredentialsError". Was the previous bucket public and the new one isn't? I'm afraid I don't know enough about AWS to solve this.

@andersy005
Copy link
Member

@aulemahal, @mgrover1 fixed the kerchunk issue in

However, the netCDF's thread safety issue is still outstanding.

@mgrover1
Copy link
Collaborator

@andersy005 - it appears to be netcdf-c issue

Unidata/netcdf4-python#1192 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants