Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Arbitrary fill value #143

Closed
hameerabbasi opened this issue Apr 23, 2018 · 2 comments
Closed

Arbitrary fill value #143

hameerabbasi opened this issue Apr 23, 2018 · 2 comments

Comments

@hameerabbasi
Copy link
Collaborator

As requested in #1.

@hameerabbasi
Copy link
Collaborator Author

@shoyer How important is this to XArray? While I find that implementing fill values is trivial algorithmically, a few lines go into every single function in the whole library, and may lead to a lot of corner cases and bugs. I'm debating whether it's worth it to implement this in practice.

@shoyer
Copy link
Member

shoyer commented Apr 28, 2018

I do think that sparse arrays in xarray be a very interesting alternative to using MultiIndex in pandas, especially when coupled with tools like dask for scaling to large datasets.

For this to work in any reasonable way, support for NaN/NA as a fill-value will be much pretty essential.
Most xarray operations automatically insert or skip NaN values (in data alignment/aggregations), so NaN is much more natural and useful fill-value than zero. Pandas's sparse data-structures support flexible fill-values defaulting to NaN for this same reason.

For computation oriented use-cases, I'm sure sparse would get a bit of use with xarray even if fill-values were always zero. But for analytics (which I think is the most exciting use-case), fill-values of NaN would be pretty much essential.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants