Skip to content

Commit

Permalink
cleanup based on comments
Browse files Browse the repository at this point in the history
  • Loading branch information
jreback committed Dec 19, 2015
1 parent 05eb20f commit 1890a88
Show file tree
Hide file tree
Showing 8 changed files with 355 additions and 235 deletions.
35 changes: 17 additions & 18 deletions doc/source/computation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,7 @@ accept the following arguments:
- ``window``: size of moving window
- ``min_periods``: threshold of non-null data points to require (otherwise
result is NA)
- ``center``: boolean, whether to set the labels at the center (default is False)

.. warning::

Expand Down Expand Up @@ -334,7 +335,7 @@ The following methods are available:
:meth:`~Window.sum`, Sum of values
:meth:`~Window.mean`, Mean of values

The weights used in the window are specified by the ``win_type``keyword. The list of recognized types are:
The weights used in the window are specified by the ``win_type`` keyword. The list of recognized types are:

- ``boxcar``
- ``triang``
Expand Down Expand Up @@ -370,27 +371,12 @@ For some windowing functions, additional parameters must be specified:
ser.rolling(window=5, win_type='gaussian').mean(std=0.1)
Centering Windows
~~~~~~~~~~~~~~~~~

By default the labels are set to the right edge of the window, but a
``center`` keyword is available so the labels can be set at the center.
This keyword is available in other rolling functions as well.

.. ipython:: python
ser.rolling(window=5, win_type='boxcar').mean()
ser.rolling(window=5, win_type='boxcar', center=True).mean()
ser.rolling(window=5, center=True).mean()
.. _stats.moments.normalization:

.. note::

For ``.sum()`` with a ``win_type``, there is no normalization done to the
weights. Passing custom weights of ``[1, 1, 1]`` will yield a different
weights for the window. Passing custom weights of ``[1, 1, 1]`` will yield a different
result than passing weights of ``[2, 2, 2]``, for example. When passing a
``win_type`` instead of explicitly specifying the weights, the weights are
already normalized so that the largest weight is 1.
Expand All @@ -399,6 +385,18 @@ This keyword is available in other rolling functions as well.
such that the weights are normalized with respect to each other. Weights
of ``[1, 1, 1]`` and ``[2, 2, 2]`` yield the same result.

Centering Windows
~~~~~~~~~~~~~~~~~

By default the labels are set to the right edge of the window, but a
``center`` keyword is available so the labels can be set at the center.
This keyword is available in other rolling functions as well.

.. ipython:: python
ser.rolling(window=5).mean()
ser.rolling(window=5, center=True).mean()
.. _stats.moments.binary:

Binary Window Functions
Expand Down Expand Up @@ -550,7 +548,7 @@ Furthermore you can pass a nested dict to indicate different aggregations on dif

.. ipython:: python
r.agg({'A' : {'ra' : 'sum'}, 'B' : {'rb' : 'std' }})
r.agg({'A' : ['sum','std'], 'B' : ['mean','std'] })
.. _stats.moments.expanding:
Expand Down Expand Up @@ -607,6 +605,7 @@ all accept are:
- ``min_periods``: threshold of non-null data points to require. Defaults to
minimum needed to compute statistic. No ``NaNs`` will be output once
``min_periods`` non-null data points have been seen.
- ``center``: boolean, whether to set the labels at the center (default is False)

.. note::

Expand Down
15 changes: 11 additions & 4 deletions doc/source/whatsnew/v0.14.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -170,11 +170,18 @@ API changes
:ref:`Computing rolling pairwise covariances and correlations
<stats.moments.corr_pairwise>` in the docs.

.. ipython:: python
.. code-block:: python

In [1]: df = DataFrame(np.random.randn(10,4),columns=list('ABCD'))

In [4]: covs = pd.rolling_cov(df[['A','B','C']], df[['B','C','D']], 5, pairwise=True)

df = DataFrame(np.random.randn(10,4),columns=list('ABCD'))
covs = rolling_cov(df[['A','B','C']], df[['B','C','D']], 5, pairwise=True)
covs[df.index[-1]]
In [5]: covs[df.index[-1]]
Out[5]:
B C D
A 0.035310 0.326593 -0.505430
B 0.137748 -0.006888 -0.005383
C -0.006888 0.861040 0.020762

- ``Series.iteritems()`` is now lazy (returns an iterator rather than a list). This was the documented behavior prior to 0.14. (:issue:`6760`)

Expand Down
103 changes: 80 additions & 23 deletions doc/source/whatsnew/v0.15.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ For full docs, see the :ref:`categorical introduction <categorical>` and the

.. ipython:: python
:okwarning:

df = DataFrame({"id":[1,2,3,4,5,6], "raw_grade":['a', 'b', 'b', 'a', 'a', 'e']})

df["grade"] = df["raw_grade"].astype("category")
Expand Down Expand Up @@ -353,9 +353,15 @@ Rolling/Expanding Moments improvements

New behavior

.. ipython:: python
.. code-block:: python

rolling_min(s, window=10, min_periods=5)
In [4]: pd.rolling_min(s, window=10, min_periods=5)
Out[4]:
0 NaN
1 NaN
2 NaN
3 NaN
dtype: float64

- :func:`rolling_max`, :func:`rolling_min`, :func:`rolling_sum`, :func:`rolling_mean`, :func:`rolling_median`,
:func:`rolling_std`, :func:`rolling_var`, :func:`rolling_skew`, :func:`rolling_kurt`, :func:`rolling_quantile`,
Expand All @@ -381,9 +387,15 @@ Rolling/Expanding Moments improvements

New behavior (note final value is ``5 = sum([2, 3, NaN])``):

.. ipython:: python
.. code-block:: python

rolling_sum(Series(range(4)), window=3, min_periods=0, center=True)
In [7]: rolling_sum(Series(range(4)), window=3, min_periods=0, center=True)
Out[7]:
0 1
1 3
2 6
3 5
dtype: float64

- :func:`rolling_window` now normalizes the weights properly in rolling mean mode (`mean=True`) so that
the calculated weighted means (e.g. 'triang', 'gaussian') are distributed about the same means as those
Expand All @@ -397,20 +409,27 @@ Rolling/Expanding Moments improvements

.. code-block:: python

In [39]: rolling_window(s, window=3, win_type='triang', center=True)
Out[39]:
0 NaN
1 6.583333
2 6.883333
3 6.683333
4 NaN
dtype: float64
In [39]: rolling_window(s, window=3, win_type='triang', center=True)
Out[39]:
0 NaN
1 6.583333
2 6.883333
3 6.683333
4 NaN
dtype: float64

New behavior

.. ipython:: python

rolling_window(s, window=3, win_type='triang', center=True)
In [10]: pd.rolling_window(s, window=3, win_type='triang', center=True)
Out[10]:
0 NaN
1 9.875
2 10.325
3 10.025
4 NaN
dtype: float64

- Removed ``center`` argument from all :func:`expanding_ <expanding_apply>` functions (see :ref:`list <api.functions_expanding>`),
as the results produced when ``center=True`` did not make much sense. (:issue:`7925`)
Expand Down Expand Up @@ -449,9 +468,17 @@ Rolling/Expanding Moments improvements

New behavior (note values start at index ``4``, the location of the 2nd (since ``min_periods=2``) non-empty value):

.. ipython:: python
.. code-block:: python

ewma(s, com=3., min_periods=2)
In [2]: pd.ewma(s, com=3., min_periods=2)
Out[2]:
0 NaN
1 NaN
2 NaN
3 NaN
4 1.759644
5 2.383784
dtype: float64

- :func:`ewmstd`, :func:`ewmvol`, :func:`ewmvar`, :func:`ewmcov`, and :func:`ewmcorr`
now have an optional ``adjust`` argument, just like :func:`ewma` does,
Expand All @@ -465,11 +492,28 @@ Rolling/Expanding Moments improvements
When ``ignore_na=True`` (which reproduces the pre-0.15.0 behavior), missing values are ignored in the weights calculation.
(:issue:`7543`)

.. ipython:: python
.. code-block:: python

In [7]: pd.ewma(Series([None, 1., 8.]), com=2.)
Out[7]:
0 NaN
1 1.0
2 5.2
dtype: float64

In [8]: pd.ewma(Series([1., None, 8.]), com=2., ignore_na=True) # pre-0.15.0 behavior
Out[8]:
0 1.0
1 1.0
2 5.2
dtype: float64

ewma(Series([None, 1., 8.]), com=2.)
ewma(Series([1., None, 8.]), com=2., ignore_na=True) # pre-0.15.0 behavior
ewma(Series([1., None, 8.]), com=2., ignore_na=False) # new default
In [9]: pd.ewma(Series([1., None, 8.]), com=2., ignore_na=False) # new default
Out[9]:
0 1.000000
1 1.000000
2 5.846154
dtype: float64

.. warning::

Expand Down Expand Up @@ -525,10 +569,23 @@ Rolling/Expanding Moments improvements
By comparison, the following 0.15.0 results have a ``NaN`` for entry ``0``,
and the debiasing factors are decreasing (towards 1.25):

.. ipython:: python
.. code-block:: python

ewmvar(s, com=2., bias=False)
ewmvar(s, com=2., bias=False) / ewmvar(s, com=2., bias=True)
In [14]: pd.ewmvar(s, com=2., bias=False)
Out[14]:
0 NaN
1 0.500000
2 1.210526
3 4.089069
dtype: float64

In [15]: pd.ewmvar(s, com=2., bias=False) / pd.ewmvar(s, com=2., bias=True)
Out[15]:
0 NaN
1 2.083333
2 1.583333
3 1.425439
dtype: float64

See :ref:`Exponentially weighted moment functions <stats.moments.exponentially_weighted>` for details. (:issue:`7912`)

Expand Down
12 changes: 7 additions & 5 deletions doc/source/whatsnew/v0.18.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,8 @@ And multiple aggregations

.. ipython:: python

r.agg({'A' : {'ra' : ['mean','std']},
'B' : {'rb' : ['mean','std']}})
r.agg({'A' : ['mean','std'],
'B' : ['mean','std']})

.. _whatsnew_0180.enhancements.other:

Expand Down Expand Up @@ -239,15 +239,17 @@ Deprecations
2 0.5
dtype: float64

- The the ``freq`` and ``how`` arguments to the ``.rolling``, ``.expanding``, and ``.ewm`` (new) functions are deprecated, and will be removed in a future version. (:issue:`11603`)
- The the ``freq`` and ``how`` arguments to the ``.rolling``, ``.expanding``, and ``.ewm`` (new) functions are deprecated, and will be removed in a future version. You can simply resample the input prior to creating a window function. (:issue:`11603`).

For example, instead of ``s.rolling(window=5,freq='D').max()`` to get the max value on a rolling 5 Day window, one could use ``s.resample('D',how='max').rolling(window=5).max()``, which first resamples the data to daily data, then provides a rolling 5 day window.

.. _whatsnew_0180.prior_deprecations:

Removal of prior version deprecations/changes
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

- Removal of ``rolling_corr_parwise`` in favor of ``.rolling().corr(pairwise=True)`` (:issue:`4950`)
- Removal of ``expanding_corr_parwise`` in favor of ``.expanding().corr(pairwise=True)`` (:issue:`4950`)
- Removal of ``rolling_corr_pairwise`` in favor of ``.rolling().corr(pairwise=True)`` (:issue:`4950`)
- Removal of ``expanding_corr_pairwise`` in favor of ``.expanding().corr(pairwise=True)`` (:issue:`4950`)



Expand Down
15 changes: 9 additions & 6 deletions pandas/core/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -358,11 +358,10 @@ def _gotitem(self, key, ndim, subset=None):
"""

_see_also_template = """
See also
--------
`pandas.Series.%(name)s`
`pandas.DataFrame.%(name)s`
pandas.Series.%(name)s
pandas.DataFrame.%(name)s
"""

def aggregate(self, func, *args, **kwargs):
Expand Down Expand Up @@ -422,7 +421,7 @@ def _aggregate(self, arg, *args, **kwargs):
else:
for col, agg_how in compat.iteritems(arg):
colg = self._gotitem(col, ndim=1)
result[col] = colg.aggregate(agg_how, _level=(_level or 0) + 1)
result[col] = colg.aggregate(agg_how, _level=None)
keys.append(col)

if isinstance(list(result.values())[0], com.ABCDataFrame):
Expand Down Expand Up @@ -451,12 +450,16 @@ def _aggregate_multiple_funcs(self, arg, _level):
if self.axis != 0:
raise NotImplementedError("axis other than 0 is not supported")

obj = self._obj_with_exclusions
if self._selected_obj.ndim == 1:
obj = self._selected_obj
else:
obj = self._obj_with_exclusions

results = []
keys = []

# degenerate case
if obj.ndim == 1:
if obj.ndim==1:
for a in arg:
try:
colg = self._gotitem(obj.name, ndim=1, subset=obj)
Expand Down
Loading

0 comments on commit 1890a88

Please sign in to comment.