Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API: Restore implicit converter registration #18307

Merged
merged 26 commits into from
Dec 7, 2017
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion ci/check_imports.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@
'ipython',
'jinja2'
'lxml',
'matplotlib',
'numexpr',
'openpyxl',
'py',
Expand Down
46 changes: 45 additions & 1 deletion doc/source/whatsnew/v0.21.1.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,50 @@ This is a minor release from 0.21.1 and includes a number of deprecations, new
features, enhancements, and performance improvements along with a large number
of bug fixes. We recommend that all users upgrade to this version.

.. _whatsnew_0211.special:

Restore Matplotlib datetime Converter Registration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Pandas implements some matplotlib converters for nicely formatting the axis
labels on plots with ``datetime`` or ``Period`` values. Prior to pandas 0.21.0,
these were implicitly registered with matplotlib, as a side effect of
``import pandas``. In pandas 0.21.0, we required users to explicitly register
the converter.

.. code-block:: python

>>> from pandas.tseries import converter
>>> converter.register()

This caused problems for some users, so we're temporarily reverting that change;
pandas will again register the converters on import. Using the converters
without explicitly registering the converters will cause a ``FutureWarning``:

.. code-block:: python

>>> import pandas as pd
>>> import matplotlib.pyplot as plt
>>> fig, ax = plt.subplots()
>>> ax.plot(pd.Series(range(12), index=pd.date_range('2017', periods=12)))
FutureWarning: Using an implicitly registered datetime converter for a
matplotlib plotting method. The converter was registered by pandas on import.
Future versions of pandas will require you to explicitly register matplotlib
converters.

To register the converters:
>>> from pandas.tseries import converter
>>> converter.register()

As the error message says, you'll need to register the converters if you intend
to use them with matplotlib plotting functions. Pandas plotting functions, such
as ``Series.plot``, will register them for you calling ``converter.register()``
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"for you calling" -> "for you, and calling"

first is not necessary.

Finally, control the formatters, we've added a new option:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"to" control

``pd.options.plotting.matplotlib.register_formatters``. By default, they are
registered. Toggling this to ``False`` removes pandas' formatters (:issue:`18301`)

.. _whatsnew_0211.enhancements:

New features
Expand All @@ -30,7 +74,7 @@ Other Enhancements
Deprecations
~~~~~~~~~~~~

-
-
-
-

Expand Down
28 changes: 28 additions & 0 deletions pandas/core/config_init.py
Original file line number Diff line number Diff line change
Expand Up @@ -480,3 +480,31 @@ def use_inf_as_na_cb(key):
cf.register_option(
'engine', 'auto', parquet_engine_doc,
validator=is_one_of_factory(['auto', 'pyarrow', 'fastparquet']))

# --------
# Plotting
# ---------

register_formatter_doc = """
: bool
Whether to register formatters with matplotlib's units registry for
dates, times, datetimes, and Periods.
"""


def register_formatter_cb(key):
from matplotlib import units
from pandas.plotting._converter import (
register, get_pairs)

if cf.get_option(key):
register()
else:
for type_, cls in get_pairs():
if isinstance(units.registry.get(type_), cls):
units.registry.pop(type_)


with cf.config_prefix("plotting.matplotlib"):
cf.register_option("register_formatters", True, register_formatter_doc,
validator=bool, cb=register_formatter_cb)
4 changes: 4 additions & 0 deletions pandas/plotting/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,7 @@
from pandas.plotting._core import boxplot
from pandas.plotting._style import plot_params
from pandas.plotting._tools import table
try:
from pandas.plotting import _converter as converter
except ImportError:
pass
53 changes: 46 additions & 7 deletions pandas/plotting/_converter.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import warnings
from datetime import datetime, timedelta
import datetime as pydt
import numpy as np
Expand Down Expand Up @@ -45,14 +46,42 @@

MUSEC_PER_DAY = 1e6 * SEC_PER_DAY

_WARN = True

def register():
units.registry[lib.Timestamp] = DatetimeConverter()
units.registry[Period] = PeriodConverter()
units.registry[pydt.datetime] = DatetimeConverter()
units.registry[pydt.date] = DatetimeConverter()
units.registry[pydt.time] = TimeConverter()
units.registry[np.datetime64] = DatetimeConverter()
def get_pairs():
return [
(lib.Timestamp, DatetimeConverter),
(Period, PeriodConverter),
(pydt.datetime, DatetimeConverter),
(pydt.date, DatetimeConverter),
(pydt.time, TimeConverter),
(np.datetime64, DatetimeConverter),
]


def register(warn=False):
global _WARN

if not warn:
_WARN = False

for type_, cls in get_pairs():
units.registry[type_] = cls()


def _check_implicitly_registered():
global _WARN

if _WARN:
msg = ("Using an implicitly registered datetime converter for a "
"matplotlib plotting method. The converter was registered "
"by pandas on import. Future versions of pandas will require "
"you to explicitly register matplotlib converters.\n\n"
"To register the converters:\n\t"
">>> from pandas.tseries import converter\n\t"
">>> converter.register()")
warnings.warn(msg, FutureWarning)
_WARN = False


def _to_ordinalf(tm):
Expand Down Expand Up @@ -190,6 +219,7 @@ class DatetimeConverter(dates.DateConverter):
@staticmethod
def convert(values, unit, axis):
# values might be a 1-d array, or a list-like of arrays.
_check_implicitly_registered()
if is_nested_list_like(values):
values = [DatetimeConverter._convert_1d(v, unit, axis)
for v in values]
Expand Down Expand Up @@ -274,6 +304,7 @@ class PandasAutoDateLocator(dates.AutoDateLocator):

def get_locator(self, dmin, dmax):
'Pick the best locator based on a distance.'
_check_implicitly_registered()
delta = relativedelta(dmax, dmin)

num_days = (delta.years * 12.0 + delta.months) * 31.0 + delta.days
Expand Down Expand Up @@ -315,6 +346,7 @@ def get_unit_generic(freq):

def __call__(self):
# if no data have been set, this will tank with a ValueError
_check_implicitly_registered()
try:
dmin, dmax = self.viewlim_to_dt()
except ValueError:
Expand Down Expand Up @@ -917,6 +949,8 @@ def _get_default_locs(self, vmin, vmax):
def __call__(self):
'Return the locations of the ticks.'
# axis calls Locator.set_axis inside set_m<xxxx>_formatter
_check_implicitly_registered()

vi = tuple(self.axis.get_view_interval())
if vi != self.plot_obj.view_interval:
self.plot_obj.date_axis_info = None
Expand Down Expand Up @@ -1001,6 +1035,8 @@ def set_locs(self, locs):
'Sets the locations of the ticks'
# don't actually use the locs. This is just needed to work with
# matplotlib. Force to use vmin, vmax
_check_implicitly_registered()

self.locs = locs

(vmin, vmax) = vi = tuple(self.axis.get_view_interval())
Expand All @@ -1012,6 +1048,8 @@ def set_locs(self, locs):
self._set_default_format(vmin, vmax)

def __call__(self, x, pos=0):
_check_implicitly_registered()

if self.formatdict is None:
return ''
else:
Expand Down Expand Up @@ -1042,6 +1080,7 @@ def format_timedelta_ticks(x, pos, n_decimals):
return s

def __call__(self, x, pos=0):
_check_implicitly_registered()
(vmin, vmax) = tuple(self.axis.get_view_interval())
n_decimals = int(np.ceil(np.log10(100 * 1e9 / (vmax - vmin))))
if n_decimals > 9:
Expand Down
28 changes: 14 additions & 14 deletions pandas/plotting/_core.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@

from pandas.util._decorators import cache_readonly
from pandas.core.base import PandasObject
from pandas.core.config import get_option
from pandas.core.dtypes.missing import isna, notna, remove_na_arraylike
from pandas.core.dtypes.common import (
is_list_like,
Expand Down Expand Up @@ -40,16 +41,13 @@
_get_xlim, _set_ticks_props,
format_date_labels)

_registered = False


def _setup():
# delay the import of matplotlib until nescessary
global _registered
if not _registered:
from pandas.plotting import _converter
_converter.register()
_registered = True
try:
from pandas.plotting import _converter
except ImportError:
pass
else:
if get_option('plotting.matplotlib.register_formatters'):
_converter.register(warn=True)


def _get_standard_kind(kind):
Expand Down Expand Up @@ -99,7 +97,7 @@ def __init__(self, data, kind=None, by=None, subplots=False, sharex=None,
secondary_y=False, colormap=None,
table=False, layout=None, **kwds):

_setup()
_converter._WARN = False
self.data = data
self.by = by

Expand Down Expand Up @@ -2059,7 +2057,7 @@ def boxplot_frame(self, column=None, by=None, ax=None, fontsize=None, rot=0,
grid=True, figsize=None, layout=None,
return_type=None, **kwds):
import matplotlib.pyplot as plt
_setup()
_converter._WARN = False
ax = boxplot(self, column=column, by=by, ax=ax, fontsize=fontsize,
grid=grid, rot=rot, figsize=figsize, layout=layout,
return_type=return_type, **kwds)
Expand Down Expand Up @@ -2155,7 +2153,7 @@ def hist_frame(data, column=None, by=None, grid=True, xlabelsize=None,
kwds : other plotting keyword arguments
To be passed to hist function
"""
_setup()
_converter._WARN = False
if by is not None:
axes = grouped_hist(data, column=column, by=by, ax=ax, grid=grid,
figsize=figsize, sharex=sharex, sharey=sharey,
Expand Down Expand Up @@ -2289,6 +2287,8 @@ def grouped_hist(data, column=None, by=None, ax=None, bins=50, figsize=None,
-------
axes: collection of Matplotlib Axes
"""
_converter._WARN = False

def plot_group(group, ax):
ax.hist(group.dropna().values, bins=bins, **kwargs)

Expand Down Expand Up @@ -2352,7 +2352,7 @@ def boxplot_frame_groupby(grouped, subplots=True, column=None, fontsize=None,
>>> grouped = df.unstack(level='lvl1').groupby(level=0, axis=1)
>>> boxplot_frame_groupby(grouped, subplots=False)
"""
_setup()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this one doesn't need to be replaced with _converter.WARN = False ?

_converter._WARN = False
if subplots is True:
naxes = len(grouped)
fig, axes = _subplots(naxes=naxes, squeeze=False,
Expand Down
72 changes: 71 additions & 1 deletion pandas/tests/plotting/test_converter.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,9 @@
from datetime import datetime, date

import numpy as np
from pandas import Timestamp, Period, Index
from pandas import Timestamp, Period, Index, date_range, Series
from pandas.compat import u
import pandas.core.config as cf
import pandas.util.testing as tm
from pandas.tseries.offsets import Second, Milli, Micro, Day
from pandas.compat.numpy import np_datetime64_compat
Expand All @@ -15,6 +16,75 @@ def test_timtetonum_accepts_unicode():
assert (converter.time2num("00:01") == converter.time2num(u("00:01")))


class TestRegistration(object):

def test_warns(self):
plt = pytest.importorskip("matplotlib.pyplot")
s = Series(range(12), index=date_range('2017', periods=12))
_, ax = plt.subplots()

# Set to the "warning" state, in case this isn't the first test run
converter._WARN = True
with tm.assert_produces_warning(FutureWarning,
check_stacklevel=False) as w:
ax.plot(s.index, s.values)
plt.close()

assert len(w) == 1
assert "Using an implicitly registered datetime converter" in str(w[0])

def test_registering_no_warning(self):
plt = pytest.importorskip("matplotlib.pyplot")
s = Series(range(12), index=date_range('2017', periods=12))
_, ax = plt.subplots()

# Set to the "warn" state, in case this isn't the first test run
converter._WARN = True
converter.register()
with tm.assert_produces_warning(None) as w:
ax.plot(s.index, s.values)

assert len(w) == 0

def test_pandas_plots_register(self):
pytest.importorskip("matplotlib.pyplot")
s = Series(range(12), index=date_range('2017', periods=12))
# Set to the "warn" state, in case this isn't the first test run
converter._WARN = True
with tm.assert_produces_warning(None) as w:
s.plot()

assert len(w) == 0

def test_matplotlib_formatters(self):
units = pytest.importorskip("matplotlib.units")
assert Timestamp in units.registry

ctx = cf.option_context("plotting.matplotlib.register_formatters",
False)
with ctx:
assert Timestamp not in units.registry

assert Timestamp in units.registry

def test_option_no_warning(self):
pytest.importorskip("matplotlib.pyplot")
ctx = cf.option_context("plotting.matplotlib.register_formatters",
False)
plt = pytest.importorskip("matplotlib.pyplot")
s = Series(range(12), index=date_range('2017', periods=12))
_, ax = plt.subplots()

# Set to the "warn" state, in case this isn't the first test run
converter._WARN = True
converter.register()
with ctx:
with tm.assert_produces_warning(None) as w:
ax.plot(s.index, s.values)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you did set the option to False, it should not warn?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, forget, that is what is being tested :-) (and they have been registered manually afterwards)

Should we test what happens if you do not register manually after you set the option?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean set the option to True or False? I think my latest commit does that.

After setting it to False, I think we should not warn even if they don't register manually (which is what my tests checks hopefully)


assert len(w) == 0


class TestDateTimeConverter(object):

def setup_method(self, method):
Expand Down