Skip to content

Commit

Permalink
ENH: add time-window capability to .rolling
Browse files Browse the repository at this point in the history
xref pandas-dev#13327
CLN: pep for cython, xref pandas-dev#12995
  • Loading branch information
jreback committed Jun 25, 2016
1 parent bf4786a commit 05331e8
Show file tree
Hide file tree
Showing 10 changed files with 2,721 additions and 649 deletions.
16 changes: 14 additions & 2 deletions ci/lint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ source activate pandas
RET=0

if [ "$LINT" ]; then
echo "Linting"
echo "Linting *.py"
for path in 'core' 'indexes' 'types' 'formats' 'io' 'stats' 'compat' 'sparse' 'tools' 'tseries' 'tests' 'computation' 'util'
do
echo "linting -> pandas/$path"
Expand All @@ -17,7 +17,19 @@ if [ "$LINT" ]; then
fi

done
echo "Linting DONE"
echo "Linting *.py DONE"

echo "Linting *.pyx"
for path in 'window.pyx'
do
echo "linting -> pandas/$path"
flake8 pandas/$path --filename '*.pyx' --select=E501,E302,E203,E226,E111,E114,E221,E303,E128,E231,E126,E128
if [ $? -ne "0" ]; then
RET=1
fi

done
echo "Linting *.pyx DONE"

echo "Check for invalid testing"
grep -r -E --include '*.py' --exclude nosetester.py --exclude testing.py '(numpy|np)\.testing' pandas
Expand Down
61 changes: 61 additions & 0 deletions doc/source/computation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -391,6 +391,67 @@ For some windowing functions, additional parameters must be specified:
such that the weights are normalized with respect to each other. Weights
of ``[1, 1, 1]`` and ``[2, 2, 2]`` yield the same result.

.. _stats.moments.ts:

Time-aware Rolling
~~~~~~~~~~~~~~~~~~

.. versionadded:: 0.18.2

New in version 0.18.2 are the ability to pass an offset (or convertible) to a ``.rolling()`` method and have it produce
variable sized windows based on the passed time window. This is useful for a non-regular time frequency index.

.. ipython:: python
dft = pd.DataFrame({'B': [0, 1, 2, np.nan, 4]},
index=pd.date_range('20130101 09:00:00', periods=5))
dft
This is a regular frequency index. Using an integer window parameter works to roll along the window frequency.

.. ipython:: python
dft.rolling(2).sum()
dft.rolling(2, min_periods=1).sum()
Specifying an offset allows a more intuitive specification of the rolling frequency.

.. ipython:: python
dft.rolling('2s').sum()
Using a non-regular, but still monotonic index, rolling with an integer window does not impart any special calculation.

.. ipython:: python
dft = DataFrame({'B': [0, 1, 2, np.nan, 4]},
index = pd.Index([pd.Timestamp('20130101 09:00:00'),
pd.Timestamp('20130101 09:00:02'),
pd.Timestamp('20130101 09:00:03'),
pd.Timestamp('20130101 09:00:05'),
pd.Timestamp('20130101 09:00:06')],
name='foo'))
dft
dft.rolling(2).sum()
Using the time-specification generates variable windows for this sparse data.

.. ipython:: python
dft.rolling('2s').sum()
Furthermore, we now allow an optional ``on`` parameter to specify a column (rather than the
default of the index) in a DataFrame.

.. ipython:: python
dft = dft.reset_index()
dft
dft.rolling('2s', on='foo').sum()
Centering Windows
~~~~~~~~~~~~~~~~~

Expand Down
59 changes: 59 additions & 0 deletions doc/source/whatsnew/v0.18.2.txt
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ We recommend that all users upgrade to this version.
Highlights include:

- :func:`merge_asof` for asof-style time-series joining, see :ref:`here <whatsnew_0182.enhancements.asof_merge>`
- ``.rolling()`` are now time-series aware, see :ref:`here <whatsnew_0182.enhancements.rolling_ts>`

.. contents:: What's new in v0.18.2
:local:
Expand Down Expand Up @@ -111,6 +112,64 @@ that forward filling happens automatically taking the most recent non-NaN value.
This returns a merged DataFrame with the entries in the same order as the original left
passed DataFrame (``trades`` in this case), with the fields of the ``quotes`` merged.

.. _whatsnew_0182.enhancements.rolling_ts:

``.rolling()`` are now time-series aware
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

``.rolling()`` objects are now time-series aware and can accept a time-series offset (or convertible) for the ``window`` argument (:issue:`13327`, :issue:`12995`)
See the full documenation :ref:`here <stats.moments.ts>`.

.. ipython:: python

dft = pd.DataFrame({'B': [0, 1, 2, np.nan, 4]},
index=pd.date_range('20130101 09:00:00', periods=5))
dft

This is a regular frequency index. Using an integer window parameter works to roll along the window frequency.

.. ipython:: python

dft.rolling(2).sum()
dft.rolling(2, min_periods=1).sum()

Specifying an offset allows a more intuitive specification of the rolling frequency.

.. ipython:: python

dft.rolling('2s').sum()

Using a non-regular, but still monotonic index, rolling with an integer window does not impart any special calculation.

.. ipython:: python


dft = DataFrame({'B': [0, 1, 2, np.nan, 4]},
index = pd.Index([pd.Timestamp('20130101 09:00:00'),
pd.Timestamp('20130101 09:00:02'),
pd.Timestamp('20130101 09:00:03'),
pd.Timestamp('20130101 09:00:05'),
pd.Timestamp('20130101 09:00:06')],
name='foo'))

dft
dft.rolling(2).sum()

Using the time-specification generates variable windows for this sparse data.

.. ipython:: python

dft.rolling('2s').sum()

Furthermore, we now allow an optional ``on`` parameter to specify a column (rather than the
default of the index) in a DataFrame.

.. ipython:: python

dft = dft.reset_index()
dft
dft.rolling('2s', on='foo').sum()

.. _whatsnew_0182.enhancements.read_csv_dupe_col_names_support:

:func:`read_csv` has improved support for duplicate column names
Expand Down
5 changes: 3 additions & 2 deletions pandas/core/generic.py
Original file line number Diff line number Diff line change
Expand Up @@ -5306,11 +5306,12 @@ def _add_series_or_dataframe_operations(cls):

@Appender(rwindow.rolling.__doc__)
def rolling(self, window, min_periods=None, freq=None, center=False,
win_type=None, axis=0):
win_type=None, on=None, axis=0):
axis = self._get_axis_number(axis)
return rwindow.rolling(self, window=window,
min_periods=min_periods, freq=freq,
center=center, win_type=win_type, axis=axis)
center=center, win_type=win_type,
on=on, axis=axis)

cls.rolling = rolling

Expand Down
Loading

0 comments on commit 05331e8

Please sign in to comment.