Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

string coordinate gets converted to object coordinate upon addition of variable to dataset #627

Closed
anntzer opened this issue Oct 16, 2015 · 10 comments
Labels

Comments

@anntzer
Copy link
Contributor

anntzer commented Oct 16, 2015

With the current HEAD, consider

import numpy as np
from xray import *

ds = Dataset({"1": DataArray(np.zeros(3), dims=["a"], coords={"a": list("xyz")})})
print(ds)
ds["2"] = DataArray(np.zeros(2), dims=["a"], coords={"a": list("xy")})
print(ds)

This outputs

<xray.Dataset>
Dimensions:  (a: 3)
Coordinates:
  * a        (a) <U1 'x' 'y' 'z'
Data variables:
    1        (a) float64 0.0 0.0 0.0
<xray.Dataset>
Dimensions:  (a: 3)
Coordinates:
  * a        (a) object 'x' 'y' 'z'
Data variables:
    1        (a) float64 0.0 0.0 0.0
    2        (a) float64 0.0 0.0 nan

Note that the dtype of the a coordinate got changed after the assignment.

Python3.5, numpy 1.10.1, xray master (6ea7eb2)

With 0.6.0 the coordinate is of object dtype both before and after. I forgot why I tried master but I must have had a good reason...

@shoyer shoyer added the bug label Oct 16, 2015
@shoyer
Copy link
Member

shoyer commented Oct 16, 2015

Thanks for the report -- this does seem like a bug to me. We do end up converting from fixed width strings to object quite often (to get around limitations of NumPy's strings), but adding in a variable should not change the type of any existing variables.

It's possible the best fix here will be to convert all string arrays into an object type as soon as they are loaded into xray objects.

@anntzer
Copy link
Contributor Author

anntzer commented Oct 16, 2015

object arrays look fine to me, but I don't have a strong opinion either.

@stale
Copy link

stale bot commented Jan 30, 2019

In order to maintain a list of currently relevant issues, we mark issues as stale after a period of inactivity
If this issue remains relevant, please comment here; otherwise it will be marked as closed automatically

@stale stale bot added the stale label Jan 30, 2019
@anntzer
Copy link
Contributor Author

anntzer commented Jan 30, 2019

The bug is still valid.

@max-sixty
Copy link
Collaborator

Given that this is an index, are we dependent on pandas here?

@stale stale bot closed this as completed Mar 2, 2019
@max-sixty
Copy link
Collaborator

If this isn't an upstream pandas issue, let's reopen

(I'm not sure why @stalebot closed this - looks like it's not respecting the comments?)

@anntzer
Copy link
Contributor Author

anntzer commented Mar 2, 2019

I don't know whether it's pandas or xarray's fault, but the behavior is definitely still present.

@shoyer shoyer reopened this Mar 2, 2019
@stale stale bot removed the stale label Mar 2, 2019
@andersy005
Copy link
Member

This issue appears to have been resolved:

In [1]: import xarray as xr
 
In [2]: import numpy as np

In [3]: ds = xr.Dataset({"1": xr.DataArray(np.zeros(3), dims=["a"], coords={"a": list("xyz")})})

In [4]: ds
Out[4]: 
<xarray.Dataset>
Dimensions:  (a: 3)
Coordinates:
  * a        (a) <U1 'x' 'y' 'z'
Data variables:
    1        (a) float64 0.0 0.0 0.0

In [5]: ds["2"] = xr.DataArray(np.zeros(2), dims=["a"], coords={"a": list("xy")})

In [6]: ds
Out[6]: 
<xarray.Dataset>
Dimensions:  (a: 3)
Coordinates:
  * a        (a) <U1 'x' 'y' 'z'
Data variables:
    1        (a) float64 0.0 0.0 0.0
    2        (a) float64 0.0 0.0 nan

I tested this with

xarray: 0.17.0
pandas: 1.2.3
numpy: 1.20.1

Should we close this?

@anntzer
Copy link
Contributor Author

anntzer commented Mar 27, 2021

I have not used xarray recently, so I have no opinion there.

@keewis
Copy link
Collaborator

keewis commented Mar 27, 2021

it seems this should have been closed by #4759 (bisecting reveals that as the first good commit)

@keewis keewis closed this as completed Mar 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants