open_mfdataset doesn't work on some datasets with dask 0.12

[AlexeyPechnikov]

xr.open_mfdataset('TEST.000*.nc',concat_dim='test')
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-31-4dbe17730991> in <module>()
----> 1 xr.open_mfdataset('TEST.000*.nc',concat_dim='face')

/usr/local/lib/python2.7/site-packages/xarray/backends/api.pyc in open_mfdataset(paths, chunks, concat_dim, compat, preprocess, engine, lock, data_vars, coords, **kwargs)
    551             combined = auto_combine(datasets, concat_dim=concat_dim,
    552                                     compat=compat,
--> 553                                     data_vars=data_vars, coords=coords)
    554     except ValueError:
    555         for ds in datasets:

/usr/local/lib/python2.7/site-packages/xarray/core/combine.pyc in auto_combine(datasets, concat_dim, compat, data_vars, coords)
    434         concatenated = [_auto_concat(ds, dim=dim,
    435                                      data_vars=data_vars, coords=coords)
--> 436                         for ds in grouped]
    437     else:
    438         concatenated = datasets

/usr/local/lib/python2.7/site-packages/xarray/core/combine.pyc in _auto_concat(datasets, dim, data_vars, coords)
    363                                  'explicitly')
    364             dim, = concat_dims
--> 365         return concat(datasets, dim=dim, data_vars=data_vars, coords=coords)
    366 
    367 

/usr/local/lib/python2.7/site-packages/xarray/core/combine.pyc in concat(objs, dim, data_vars, coords, compat, positions, indexers, mode, concat_over)
    118         raise TypeError('can only concatenate xarray Dataset and DataArray '
    119                         'objects, got %s' % type(first_obj))
--> 120     return f(objs, dim, data_vars, coords, compat, positions)
    121 
    122 

/usr/local/lib/python2.7/site-packages/xarray/core/combine.pyc in _dataset_concat(datasets, dim, data_vars, coords, compat, positions)
    303         if k in concat_over:
    304             vars = ensure_common_dims([ds.variables[k] for ds in datasets])
--> 305             combined = concat_vars(vars, dim, positions)
    306             insert_result_variable(k, combined)
    307 

/usr/local/lib/python2.7/site-packages/xarray/core/variable.pyc in concat(variables, dim, positions, shortcut)
   1685         return IndexVariable.concat(variables, dim, positions, shortcut)
   1686     else:
-> 1687         return Variable.concat(variables, dim, positions, shortcut)
   1688 
   1689 

/usr/local/lib/python2.7/site-packages/xarray/core/variable.pyc in concat(cls, variables, dim, positions, shortcut)
   1234             axis = first_var.get_axis_num(dim)
   1235             dims = first_var.dims
-> 1236             data = duck_array_ops.concatenate(arrays, axis=axis)
   1237             if positions is not None:
   1238                 # TODO: deprecate this option -- we don't need it for groupby

/usr/local/lib/python2.7/site-packages/xarray/core/duck_array_ops.pyc in f(*args, **kwargs)
     49             else:
     50                 module = eager_module
---> 51             return getattr(module, name)(*args, **kwargs)
     52     else:
     53         def f(data, *args, **kwargs):

/usr/local/lib/python2.7/site-packages/dask/array/core.pyc in concatenate(seq, axis)
   2176 
   2177     uc_args = list(concat(zip(seq, inds)))
-> 2178     _, seq = unify_chunks(*uc_args, warn=False)
   2179 
   2180     bds = [a.chunks for a in seq]

/usr/local/lib/python2.7/site-packages/dask/array/core.pyc in unify_chunks(*args, **kwargs)
   1898     arginds = list(partition(2, args)) # [x, ij, y, jk] -> [(x, ij), (y, jk)]
   1899 
-> 1900     nameinds = [(a.name, i) for a, i in arginds]
   1901     blockdim_dict = dict((a.name, a.chunks) for a, _ in arginds)
   1902 

AttributeError: 'numpy.ndarray' object has no attribute 'name'

But it's works for any previous xarray version.

[shoyer]

Thanks for the report. Can you share the result of xarray.show_versions()?

Dask's concatenate function is supposed to be able to handle numpy arrays, so this suggests a possible bug in dask. If you're not using dask 0.16, please try upgrading.

[AlexeyPechnikov]

Thanks, it's works:

pip2 install xarray==0.10.0
pip2 install dask==0.16.0

And it doesn't work:

pip2 install xarray==0.10.0
pip2 install dask==0.12.0

[shoyer]

Hmm, maybe we should update the minimum required version of dask in our documentation. Ideally we should be running our test suite on this earliest support version of dask.

[jhamman]

Closed via #2204