Alignment problems with f32/f64 coordinate variables

[ajwdewit]

Code sample

The code below leads to an awkward result when combining two DataArrays. Despite the fact that both d1 and d2 are defined as (11,6) arrays, a multiplication results in a (3,2) array.

import xarray as xr
import numpy as np
d1 = np.full((11,6), 2., dtype=np.float64)
d2 = np.full_like(d1, 5.0)
lonf64 = np.arange(0,1.1,0.1, dtype=np.float64)
latf64 = np.arange(0,0.6,0.1, dtype=np.float64)
lonf32 = np.arange(0,1.1,0.1, dtype=np.float32)
latf32 = np.arange(0,0.6,0.1, dtype=np.float32)
d1 = xr.DataArray(d1, coords=[('lon', lonf64),  ('lat', latf64)])
d2 = xr.DataArray(d2, coords=[('lon', lonf32),  ('lat', latf32)])

In [21]: d1.shape
Out[21]: (11, 6)

In [22]: d2.shape
Out[22]: (11, 6)

In [20]: d1 * d2
Out[20]:
<xarray.DataArray (lon: 3, lat: 2)>
array([[10., 10.],
       [10., 10.],
       [10., 10.]])
Coordinates:
  * lon      (lon) float64 0.0 0.5 1.0
  * lat      (lat) float64 0.0 0.5

Problem description

The origin of this behaviour is that xarray cannot match the coordinate variables because of differences in precision between float32/64. Although they look the same:

In [24]: d1.lat.data[1]
Out[24]: 0.1

In [25]: d2.lat.data[1]
Out[25]: 0.1

They are not:

In [27]: "%25.20f" % d1.lat.data[1]
Out[27]: '   0.10000000000000000555'

In [28]: "%25.20f" % d2.lat.data[1]
Out[28]: '   0.10000000149011611938'

Except for the coordinate values at 0.0, 0.5, 1.0 which can be both represented with full accuracy in f32 and f64:

In [30]: "%25.20f" % d1.lat.data[5]
Out[30]: '   0.50000000000000000000'

In [31]: "%25.20f" % d2.lat.data[5]
Out[31]: '   0.50000000000000000000'

Therefore the result of the multiplication is a (3,2) array because only the coordinates that match between d1 and d2 are returned.

A bit similar issue has already been raised in this issue where tolerance in alignment between coordinate variables was discussed.

Expected Output

You could argue that this behavior is the 'expected' output from this operation. However, from the "principle of least astonishment" I would argue that it would be better to either include a tolerance in matching coordinate variables, or to raise a TypeError indicating the incompatibility in the coordinate variables.

The results above were obtained with xarray 0.11.0 and numpy 1.14.2, while this behaviour did not occur in xarray version 0.10.4 with the same numpy version.

[shoyer]

I agree that it would be really nice to support tolerance in alignment (as discussed in #2217).

Xarray hasn't changed any of this behavior recently but maybe something changed upstream -- are you using the same pandas versions?

[ajwdewit]

yes, both setups used pandas 0.23.4

[dcherian]

Closing in favour of #2217