Skip to content

Commit

Permalink
Add dtype support for reduce methods. (#1841)
Browse files Browse the repository at this point in the history
* Add dtype support for reduce methods.

* Use bottleneck if dtype == values.dtype

* Change get -> pop

* Update comments.

* float128 -> float16

* ValueError -> TypeError

* update test
  • Loading branch information
fujiisoup authored and shoyer committed Jan 20, 2018
1 parent 74d8318 commit 3bd704a
Show file tree
Hide file tree
Showing 3 changed files with 28 additions and 6 deletions.
4 changes: 3 additions & 1 deletion doc/whats-new.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,11 @@ Documentation

Enhancements
~~~~~~~~~~~~
- reduce methods such as :py:func:`DataArray.sum()` now accepts ``dtype``
arguments. (:issue:`1838`)
By `Keisuke Fujii <https://github.com/fujiisoup>`_.
- Added nodatavals attribute to DataArray when using :py:func:`~xarray.open_rasterio`. (:issue:`1736`).
By `Alan Snow <https://github.com/snowman2>`_.

- :py:func:`~plot.contourf()` learned to contour 2D variables that have both a
1D co-ordinate (e.g. time) and a 2D co-ordinate (e.g. depth as a function of
time) (:issue:`1737`).
Expand Down
12 changes: 7 additions & 5 deletions xarray/core/duck_array_ops.py
Original file line number Diff line number Diff line change
Expand Up @@ -175,11 +175,11 @@ def _create_nan_agg_method(name, numeric_only=False, np_compat=False,
no_bottleneck=False, coerce_strings=False,
keep_dims=False):
def f(values, axis=None, skipna=None, **kwargs):
# ignore keyword args inserted by np.mean and other numpy aggregators
# automatically:
kwargs.pop('dtype', None)
kwargs.pop('out', None)
if kwargs.pop('out', None) is not None:
raise TypeError('`out` is not valid for {}'.format(name))

# If dtype is supplied, we use numpy's method.
dtype = kwargs.get('dtype', None)
values = asarray(values)

if coerce_strings and values.dtype.kind in 'SU':
Expand All @@ -192,14 +192,16 @@ def f(values, axis=None, skipna=None, **kwargs):
% (name, values.dtype))
nanname = 'nan' + name
if (isinstance(axis, tuple) or not values.dtype.isnative or
no_bottleneck):
no_bottleneck or
(dtype is not None and np.dtype(dtype) != values.dtype)):
# bottleneck can't handle multiple axis arguments or non-native
# endianness
if np_compat:
eager_module = npcompat
else:
eager_module = np
else:
kwargs.pop('dtype', None)
eager_module = bn
func = _dask_or_eager_func(nanname, eager_module)
using_numpy_nan_func = (eager_module is np or
Expand Down
18 changes: 18 additions & 0 deletions xarray/tests/test_dataarray.py
Original file line number Diff line number Diff line change
Expand Up @@ -1784,6 +1784,24 @@ def test_reduce(self):
dims=['x', 'y']).mean('x')
assert_equal(actual, expected)

def test_reduce_dtype(self):
coords = {'x': [-1, -2], 'y': ['ab', 'cd', 'ef'],
'lat': (['x', 'y'], [[1, 2, 3], [-1, -2, -3]]),
'c': -999}
orig = DataArray([[-1, 0, 1], [-3, 0, 3]], coords, dims=['x', 'y'])

for dtype in [np.float16, np.float32, np.float64]:
assert orig.astype(float).mean(dtype=dtype).dtype == dtype

def test_reduce_out(self):
coords = {'x': [-1, -2], 'y': ['ab', 'cd', 'ef'],
'lat': (['x', 'y'], [[1, 2, 3], [-1, -2, -3]]),
'c': -999}
orig = DataArray([[-1, 0, 1], [-3, 0, 3]], coords, dims=['x', 'y'])

with pytest.raises(TypeError):
orig.mean(out=np.ones(orig.shape))

# skip due to bug in older versions of numpy.nanpercentile
def test_quantile(self):
for q in [0.25, [0.50], [0.25, 0.75]]:
Expand Down

0 comments on commit 3bd704a

Please sign in to comment.