Add options for zarr array definition #48
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## master #48 +/- ##
==========================================
+ Coverage 94.89% 95.00% +0.10%
==========================================
Files 10 10
Lines 392 400 +8
Branches 75 78 +3
==========================================
+ Hits 372 380 +8
Misses 10 10
Partials 10 10
Continue to review full report at Codecov.
|
eric-czech
force-pushed
the
zarr_options
branch
from
d70f41d
to
830beaa
Compare
tomwhite
approved these changes
Sep 16, 2020
rechunker/api.py
Outdated
| @@ -281,14 +311,29 @@ def _setup_rechunk( | |||
| raise ValueError("Source must be a Zarr Array or Group, or a Dask Array.") | |||
|
|
|||
|
|
|||
| def _validation_options(options): | |||
There was a problem hiding this comment.
Nit: _validate_options might read better
|
Thanks @tomwhite. Let me know if there's anything else I can do to get this merged @TomAugspurger / @rabernat. |
|
Thanks. |
|
Thanks for all your contributions @eric-czech. It's great to have you involved in rechunker! |
|
Sure thing @rabernat! |
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #46
This adds options to
rechunkthat are passed to zarr create methods. The only use cases I have in mind for this are allowing for overwrites and compression, but it may be useful in other ways. Otherwise it may eventually be worth lifting those parameters into therechunksignature given that allowing any option to be passed is a bit of a footgun.Note: I had to modify one of the existing test fixtures since it was doing almost what I wanted for a compression test except that the chunks were too small for compression to take effect. I didn't know that occurred with zarr but I can't find any other explanation for why specifying a compressor makes no difference when an array is too small. I have no idea where the cutoff is but perhaps it is related to compression block sizes? Let me know if anybody understands that better.