You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
I would like to be able to use the xarray.DataArray API when the data is sparse.COO.
Instead—as mentioned at #1 (comment) —it appears I currently need to write a custom subclass of DataArray that handles cases where sparse doesn't implement a method expected by xarray.
Describe the solution you'd like
For sparse.COO.pad and the other methods mentioned in the linked comment to be implemented so that, in general, it is possible to use the xarray.DataArray API, or at least all a certain advertised set of basic functions, with sparse data.
Describe alternatives you've considered
As mentioned, I am currently handling this with wrapper code that converts an xr.DataArray to dense data, performs operations, then re-converts the data to sparse.
Another mitigation would be for the sparse documentation to describe which DataArray operations are not supported.
Some example code:
importpandasaspdimportxarrayasxrfromnumpyimportnanidx=dict(
columns=pd.Index(["x1", "x2", "x3", "x4"], name="x"),
index=pd.Index(["y1", "y2", "y3", "y4"], name="y"),
)
# Simplified example datas1=pd.DataFrame(
[
[1.0, nan, nan, nan],
[nan, nan, 2.0, nan],
[nan, nan, nan, 3.0],
[nan, 4.0, nan, nan],
],
**idx
).stack()
print(s1)
da1=xr.DataArray.from_series(s1, sparse=True)
print(da1)
# I would like to produce data like the following by using DataArray.shift()s2=pd.DataFrame(
[
[nan, 1.0, nan, nan],
[nan, nan, nan, 2.0],
[nan, nan, nan, nan],
[nan, nan, 4.0, nan],
],
**idx
).stack()
print(s2)
da2=xr.DataArray.from_series(s2)
print(da2)
# This line raises an exception, belowda1.shift(x=1)
The exception raised:
Traceback (most recent call last):
File "test.py", line 42, in <module>
da1.shift(x=1)
File "/home/khaeru/.local/lib/python3.8/site-packages/xarray/core/dataarray.py", line 3077, in shift
variable = self.variable.shift(
File "/home/khaeru/.local/lib/python3.8/site-packages/xarray/core/variable.py", line 1205, in shift
result = result._shift_one_dim(dim, count, fill_value=fill_value)
File "/home/khaeru/.local/lib/python3.8/site-packages/xarray/core/variable.py", line 1166, in _shift_one_dim
data = duck_array_ops.pad(
File "/home/khaeru/.local/lib/python3.8/site-packages/xarray/core/duck_array_ops.py", line 56, in f
return wrapped(*args, **kwargs)
File "<__array_function__ internals>", line 5, in pad
TypeError: no implementation found for 'numpy.pad' on types that implement __array_function__: [<class 'sparse._coo.core.COO'>]
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
I would like to be able to use the xarray.DataArray API when the data is sparse.COO.
Instead—as mentioned at #1 (comment) —it appears I currently need to write a custom subclass of DataArray that handles cases where sparse doesn't implement a method expected by xarray.
Describe the solution you'd like
For
sparse.COO.pad
and the other methods mentioned in the linked comment to be implemented so that, in general, it is possible to use the xarray.DataArray API, or at least all a certain advertised set of basic functions, with sparse data.Describe alternatives you've considered
As mentioned, I am currently handling this with wrapper code that converts an xr.DataArray to dense data, performs operations, then re-converts the data to sparse.
Another mitigation would be for the sparse documentation to describe which DataArray operations are not supported.
Some example code:
The exception raised:
The text was updated successfully, but these errors were encountered: