Add S3 support

[zklaus]

Checklist

• Used a personal fork of the feedstock to propose changes
• Bumped the build number (if the version is unchanged)
• Reset the build number to 0 (if the version changed)
• Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
• Ensured the license file is being packaged.

This is a straight-up port of @djhoese's PR to add S3 support, rebased onto the latest main.

[conda-forge-webservices]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

[zklaus]

@conda-forge-admin, please rerender

[djhoese]

I'll close #160 in favor of this. Note that this was based on work by others including @ocefpaf in #125 so if something is wrong with these flags I'm not sure I'm up to date on how it should work/change.

[zklaus]

I had to deactivate the tests for Zarr to make this work, but note that this seems to be mostly due to permissions. I will try to give an overview here of the failing tests and the reasons for the failures, taken from the last attempted run before switching off the tests.

The pattern is always the same: The failing tests test a bunch of stuff with Zarr files and ultimately try to write a temporary file to an AWS bucket to which we don't have write access. Here I list all the failing tests with links to the line that names the object key to which the access is denied.

• 224 - nczarr_test_run_ut_mapapi (Failed)
• 227 - nczarr_test_run_fillonlyz (Failed)
• 228 - nczarr_test_run_ncgen4 (Failed)
• 229 - nczarr_test_run_chunkcases (Failed)
• 230 - nczarr_test_run_purezarr (Failed)
• 233 - nczarr_test_run_nczarr_fill (Failed)
• 234 - nczarr_test_run_jsonconvention (Failed)
• 235 - nczarr_test_run_strings (Failed)
• 236 - nczarr_test_run_scalar (Failed)
• 237 - nczarr_test_run_nulls (Failed)
• 241 - nczarr_test_run_filter (Failed)
• 242 - nczarr_test_run_unknown (Failed)
• 243 - nczarr_test_run_specific_filters (Failed)

[zklaus]

I think this is ready as is; we might also have a discussion about how to make the tests work.

[dopplershift]

@DennisHeimbigner @WardF Can you all shed any light on those tests? Is there a correct incantation that non-Unidata folk can use to only run zarr tests for public data?

[DennisHeimbigner]

First, what actual netcdf-c version are you using? 4.9.2, I assume.

[dopplershift]

Yes, this PR is adding S3 support to the conda-forge builds for netcdf-c 4.9.2.

[DennisHeimbigner]

4.9.2 does not support S3 testing yet unless you have access to the Unidata S3 private bucket.

[zklaus]

@DennisHeimbigner, thanks, that's right. Do you object to the proposed course of action of merging the support here without re-activating the tests?

[DennisHeimbigner]

No objection on my part -- it your code. I will note that there are significant
changes in the handling of S3 testing in the next netcdf-c release, so you
may have to do this all over again.

[zklaus]

Thanks, @DennisHeimbigner, sounds good. @dopplershift, is it ok for you to move forward here?

[valeriupredoi]

hi @dopplershift just wanted to drop by thank @zklaus for this and also say we're eagerly awaiting this to be plopped in too 😃 🦅

[zklaus]

Cheers and thanks everyone! 🎉

[valeriupredoi]

awesome, very many thanks @zklaus and @dopplershift 🍺 x2