Dataset.groupby() doesn't preserve variables order

[fmaussion]

Is it intentional? I think it is rather undesirable, but maybe there is some reason for this.

[shoyer]

This is probably a bug. Usually, we're pretty careful to always use OrderedDict internally for exactly this reason. Can you give a reproducible example?

[fmaussion]

Thanks @shoyer , here's a mwe:

import xarray as xr
import numpy as np 
ds = xr.Dataset()
for vn in ['a', 'b', 'c']:
    ds[vn] = xr.DataArray(np.arange(10), dims=['t'])
ds.groupby('t').mean()

<xarray.Dataset>
Dimensions:  (t: 10)
Coordinates:
  * t        (t) int64 0 1 2 3 4 5 6 7 8 9
Data variables:
    a        (t) float64 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0
    c        (t) float64 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0
    b        (t) float64 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0

[fmaussion]

@shoyer I'd be happy to provide a fix if you want. Could you give a short pointer as to where the logic is implemented?

[shoyer]

So the tricky part here is that it's not obvious what is breaking here. One clue is that reducing doesn't seem to be necessary -- I can reproduce this just with applying an identity transform:

In [6]: identity = lambda x: x

In [7]: ds.groupby('t').apply(identity)
Out[7]:
<xarray.Dataset>
Dimensions:  (t: 10)
Coordinates:
  * t        (t) int64 0 1 2 3 4 5 6 7 8 9
Data variables:
    a        (t) int64 0 1 2 3 4 5 6 7 8 9
    c        (t) int64 0 1 2 3 4 5 6 7 8 9
    b        (t) int64 0 1 2 3 4 5 6 7 8 9

Actually, it looks like it's probably a concat bug:

In [17]: gb = ds.groupby('t')

In [18]: grouped = [v for _, v in gb]

In [20]: [list(g.data_vars) for g in grouped]
Out[20]:
[['a', 'b', 'c'],
 ['a', 'b', 'c'],
 ['a', 'b', 'c'],
 ['a', 'b', 'c'],
 ['a', 'b', 'c'],
 ['a', 'b', 'c'],
 ['a', 'b', 'c'],
 ['a', 'b', 'c'],
 ['a', 'b', 'c'],
 ['a', 'b', 'c']]

In [21]: xr.concat(grouped, dim='t')
Out[21]:
<xarray.Dataset>
Dimensions:  (t: 10)
Coordinates:
  * t        (t) int64 0 1 2 3 4 5 6 7 8 9
Data variables:
    a        (t) int64 0 1 2 3 4 5 6 7 8 9
    c        (t) int64 0 1 2 3 4 5 6 7 8 9
    b        (t) int64 0 1 2 3 4 5 6 7 8 9

[fmaussion]

Closed via #1049

[cwerner]

I am seeing something similar, but maybe this is another issue (I'm on 0.10.0rc2)?

I do get a sorted string coordinate after a groupby...

My scenario is, that I have a dataset with a coord like this:

<xarray.DataArray 'pft' (pft: 13)>
array(['TeBE_tm', 'TeBE_itm', 'TeBE_itscl', 'TeBE_tscl', 'TeBS_tm', 'TeBS_itm',
       'TeE_s', 'TeR_s', 'TeNE', 'BBS_itm', 'BE_s', 'BS_s', 'C3G'],
      dtype='|S10')
Coordinates:
  * pft      (pft) |S10 'TeBE_tm' 'TeBE_itm' 'TeBE_itscl' 'TeBE_tscl' ...

Then I create a new coordinate that I use to aggregate:

pfts = ds.coords['pft'].values.tolist()
pfts_simplified = [remove(x) for x in pfts]

ds2['pft_agg'] = xr.full_like(ds['pft'], 0)
ds2['pft_agg'][:] = pfts_simplified
ds2_agg = ds2.groupby('pft_agg').sum(dim='pft', skipna=False)
result = ds2_agg.rename({'pft_agg': 'pft'})

Then in the end I have:

<xarray.DataArray 'pft' (pft: 8)>
array(['BBS', 'B_s', 'C3G', 'TeBE', 'TeBE_scl', 'TeBS', 'TeNE', 'Te_s'], dtype=object)
Coordinates:
  * pft      (pft) object 'BBS' 'B_s' 'C3G' 'TeBE' 'TeBE_scl' 'TeBS' 'TeNE' ...

Am I missing something?

[jhamman]

@chris-b1 - I think you're seeing the issue described in #757.

[cwerner]

@jhamman Yes, indeed. Sorry to spam this old issue. I misread this one - #757 is what'm seeing.