Support for xarray

[juanfcocontreras]

This is a feature request: add support for xarray (https://xarray.pydata.org).

xarray is a package strongly inspired by pandas that adds multi-dimensional arrays (among other things). Once a dimension or time is selected, data can be converted to dataframe directly for dtale display (https://xarray.pydata.org/en/stable/generated/xarray.DataArray.to_dataframe.html)

Recently xarray added the possibility to preview the metadata (dimensions and coordinates) in jupyter:

pydata/xarray#3425

The idea could be to try something similar, and by selecting a certain dimension or time, display the data in dtale.

I know this might not be a trivial request, so I'm offering to help implement it. However, I think it's worth thinking about because xarray is a package that is becoming very popular in the scientific community.

A first approach might be:

If you want to show all the dimensions at one point in time:

data.sel(time="selected_time").to_dataframe()

If you want to show all your time data from a particular dimension:

data["dimension_name"].to_dataframe()

[aschonfeld]

@juanfcocontreras Sorry, was finishing up a couple other enhancements. Will start working on this one in this release :)

[aschonfeld]

@juanfcocontreras so how about being able to pass in an xarray to dtale.show and what it will do is keep a reference to the xr.Dataset you've passed in and by default display the entire dataframe (ds.to_dataframe()), but will also present the user with another option viewing each individual dataframe based on different dimension combinations:

• ds.sel(col1=val1, col2=val2, ... colN=valN).to_dataframe()

I have noticed that if you call to_dataframe on the output of a .sel() call when there's zero dimenstions yields the following exception: ValueError: no valid index for a 0-dimensional object

Rather than throw an exception I'll call .to_dict() instead and convert that to a dataframe with this snippet.

data = {}
for coord, coord_data in x['coords'].items():
    data[coord] = coord_data['data']
for col, col_data in x['data_vars'].items():
    data[col] = col_data['data']
pd.DataFrame([data])

Sound good?

[juanfcocontreras]

That sounds awesome! I can't wait to try it!

Thank you very much!

[aschonfeld]

@juanfcocontreras Here's what I have so far: https://youtu.be/eDg4nWGjw9k

I might try and add a reverse option where you can convert a dataframe you're viewing to an xarray dataset as well. Then you can view different dimensions. I still need to give users the ability to open a dimension in a new slice. That's not hard to add though. :)

[juanfcocontreras]

Wow, that's impressive! 👏 👏

[aschonfeld]

@juanfcocontreras just released v1.8.16 with this functionality added. Here's the docs

[juanfcocontreras]

Thank you very much. Do you know when it will be available on conda-forge? Your releases are usually almost instantaneous, but this time it's taking longer than usual.

[aschonfeld]

Should be there now

[juanfcocontreras]

I still don't see it, I guess it'll take a while to update:

https://anaconda.org/conda-forge/dtale

Thanks anyway!

[aschonfeld]

Ahhh, now i see. It looks like the conda build broke. I’ll fix it tonight. Apologies

[aschonfeld]

Alrighty, @juanfcocontreras, should be all set now. Let me know and I'll close this. Apologies for the wait.

[juanfcocontreras]

Installed and willing to try it! I'll give you feedback!

Thank you very much!