New opendap dataformat

[aulemahal]

Change Summary

Adds a new data format : "opendap".

Related issue number

Fixes #520.

Checklist

• Unit tests for the changes exist
• Tests pass on CI
• Documentation reflects the changes where applicable

Before going further in adding tests and documentation, I wanted to get some feedback on this, because I'm not sure I fully understand what fsspec does here.

[mgrover1]

This looks good - can you add a test here?

[mgrover1]

@aulemahal - can you merge the latest changes from main here? This should fix the failing docs.

[aulemahal]

Here we are!

The new test simply checks that the dataset opens with to_dask. The new test catalog lists datasets from Nasa's Hyrax Test Opendap website, specifically a list of Pathfinder's SST maps. I am not familiar with this data, I simply wanted a list of easy access opendap streams that we can play with. Thus, the catalog columns "first_swap", "last_swap" and "scode" are pure guesses as to what those mean in the file path.

Also, I added pydap to the testing environment. I am not aware of a method to see if a dataset open with netCDF4 uses OpenDap or if it was downloade on-the-fly. Whereas, pydap will just fail if the source is not an opendap stream. Also, pydap better supports opendap authentification, if we ever want to test that?

I plan on using this new catalog in #569 too, by removing the aggregation_control entry completely.