Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLN: Remove/deprecate unused/misleading dtype functions #23917

Merged
merged 25 commits into from
Nov 27, 2018
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
607f853
remove unused maybe_convert_scalar
jbrockmendel Nov 25, 2018
1e95d48
remove unused maybe_convert_string_to_object
jbrockmendel Nov 25, 2018
714c287
remove deprecated is_floating_dtype
jbrockmendel Nov 25, 2018
ba6ef3d
deprecate is_period
jbrockmendel Nov 25, 2018
a983755
deprecate is_datetimetz
jbrockmendel Nov 25, 2018
d7944b6
set stacklevel
jbrockmendel Nov 26, 2018
2c9b406
Merge branch 'master' of https://github.com/pandas-dev/pandas into dt…
jbrockmendel Nov 26, 2018
34c4e91
remove unused is_timedelta_array and is_timedelta64_array
jbrockmendel Nov 26, 2018
8e86599
remove is_int_or_datetime_dtype
jbrockmendel Nov 26, 2018
0cf6239
GH reference
jbrockmendel Nov 26, 2018
8adda17
fixup typecheck
jbrockmendel Nov 26, 2018
fe02240
typo fixu[
jbrockmendel Nov 26, 2018
53d1de4
Merge branch 'master' of https://github.com/pandas-dev/pandas into dt…
jbrockmendel Nov 26, 2018
e2f8669
Address comments
jbrockmendel Nov 26, 2018
3daaabf
isort
jbrockmendel Nov 26, 2018
2566a18
dummy commit to force CI
jbrockmendel Nov 26, 2018
8342b97
Merge branch 'master' of https://github.com/pandas-dev/pandas into dt…
jbrockmendel Nov 26, 2018
a7f7916
Merge branch 'master' of https://github.com/pandas-dev/pandas into dt…
jbrockmendel Nov 27, 2018
6dda1be
separated conditions
jbrockmendel Nov 27, 2018
9c64633
Merge branch 'master' of https://github.com/pandas-dev/pandas into dt…
jbrockmendel Nov 27, 2018
3d8752f
docstring fixup
jbrockmendel Nov 27, 2018
bb8a814
Merge branch 'master' of https://github.com/pandas-dev/pandas into dt…
jbrockmendel Nov 27, 2018
71483b5
revert usage of needs_i8_conversion
jbrockmendel Nov 27, 2018
e0593bb
dummy commit to force CI
jbrockmendel Nov 27, 2018
5bdc157
Merge branch 'master' of https://github.com/pandas-dev/pandas into dt…
jbrockmendel Nov 27, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions doc/source/whatsnew/v0.24.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1046,6 +1046,8 @@ Deprecations
- The ``keep_tz=False`` option (the default) of the ``keep_tz`` keyword of
:meth:`DatetimeIndex.to_series` is deprecated (:issue:`17832`).
- Timezone converting a tz-aware ``datetime.datetime`` or :class:`Timestamp` with :class:`Timestamp` and the ``tz`` argument is now deprecated. Instead, use :meth:`Timestamp.tz_convert` (:issue:`23579`)
- :func:`pandas.types.is_period` is deprecated in favor of `pandas.types.is_period_dtype` (:issue:`23917`)
jreback marked this conversation as resolved.
Show resolved Hide resolved
- :func:`pandas.types.is_datetimetz` is deprecated in favor of `pandas.types.is_datetime64tz` (:issue:`23917`)

.. _whatsnew_0240.deprecations.datetimelike_int_ops:

Expand Down Expand Up @@ -1125,6 +1127,7 @@ Removal of prior version deprecations/changes
- :meth:`SparseSeries.to_dense` has dropped the ``sparse_only`` parameter (:issue:`14686`)
- :meth:`DataFrame.astype` and :meth:`Series.astype` have renamed the ``raise_on_error`` argument to ``errors`` (:issue:`14967`)
- ``is_sequence``, ``is_any_int_dtype``, and ``is_floating_dtype`` have been removed from ``pandas.api.types`` (:issue:`16163`, :issue:`16189`)
- ``is_floating_dtype`` has been removed (:issue:`????`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks like a dupe of the line above

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existing whatsnew makes it look like this has already been removed, but it is still present in master, and not listed (as deprecated or removed) in #6581.

The whatsnew also says is_sequence and is_any_int_dtype have been removed, but each of these are still used in a few places internally. Get rid of those usages? (any_int_dtype only has two usages so that one will be easy)


.. _whatsnew_0240.performance:

Expand Down
31 changes: 2 additions & 29 deletions pandas/_libs/lib.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -1250,25 +1250,19 @@ def infer_dtype(value: object, skipna: bool=False) -> str:
if util.is_datetime64_object(val):
if is_datetime64_array(values):
return 'datetime64'
elif is_timedelta_or_timedelta64_array(values):
return 'timedelta'

elif is_timedelta(val):
if is_timedelta_or_timedelta64_array(values):
return 'timedelta'

elif util.is_integer_object(val):
# a timedelta will show true here as well
if is_timedelta(val):
if is_timedelta_or_timedelta64_array(values):
return 'timedelta'
# numpy timedelta64 objects are caught in the previous branch;
jbrockmendel marked this conversation as resolved.
Show resolved Hide resolved
# ordering matters here

if is_integer_array(values):
return 'integer'
elif is_integer_float_array(values):
return 'mixed-integer-float'
elif is_timedelta_or_timedelta64_array(values):
return 'timedelta'
return 'mixed-integer'

elif PyDateTime_Check(val):
Expand Down Expand Up @@ -1701,27 +1695,6 @@ cdef class TimedeltaValidator(TemporalValidator):
return is_null_timedelta64(value)


# TODO: Not used outside of tests; remove?
def is_timedelta_array(values: ndarray) -> bool:
cdef:
TimedeltaValidator validator = TimedeltaValidator(len(values),
skipna=True)
return validator.validate(values)


cdef class Timedelta64Validator(TimedeltaValidator):
cdef inline bint is_value_typed(self, object value) except -1:
return util.is_timedelta64_object(value)


# TODO: Not used outside of tests; remove?
def is_timedelta64_array(values: ndarray) -> bool:
cdef:
Timedelta64Validator validator = Timedelta64Validator(len(values),
skipna=True)
return validator.validate(values)


cdef class AnyTimedeltaValidator(TimedeltaValidator):
cdef inline bint is_value_typed(self, object value) except -1:
return is_timedelta(value)
Expand Down
4 changes: 2 additions & 2 deletions pandas/core/algorithms.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
ensure_float64, ensure_int64, ensure_object, ensure_platform_int,
ensure_uint64, is_array_like, is_bool_dtype, is_categorical_dtype,
is_complex_dtype, is_datetime64_any_dtype, is_datetime64tz_dtype,
is_datetimelike, is_datetimetz, is_extension_array_dtype, is_float_dtype,
is_datetimelike, is_extension_array_dtype, is_float_dtype,
is_integer_dtype, is_interval_dtype, is_list_like, is_numeric_dtype,
is_object_dtype, is_period_dtype, is_scalar, is_signed_integer_dtype,
is_sparse, is_timedelta64_dtype, is_unsigned_integer_dtype,
Expand Down Expand Up @@ -1581,7 +1581,7 @@ def take_nd(arr, indexer, axis=0, out=None, fill_value=np.nan, mask_info=None,
# dispatch to internal type takes
if is_extension_array_dtype(arr):
return arr.take(indexer, fill_value=fill_value, allow_fill=allow_fill)
elif is_datetimetz(arr):
elif is_datetime64tz_dtype(arr):
return arr.take(indexer, fill_value=fill_value, allow_fill=allow_fill)
elif is_interval_dtype(arr):
return arr.take(indexer, fill_value=fill_value, allow_fill=allow_fill)
Expand Down
44 changes: 8 additions & 36 deletions pandas/core/dtypes/cast.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,11 @@
ensure_int8, ensure_int16, ensure_int32, ensure_int64, ensure_object,
is_bool, is_bool_dtype, is_categorical_dtype, is_complex, is_complex_dtype,
is_datetime64_dtype, is_datetime64_ns_dtype, is_datetime64tz_dtype,
is_datetime_or_timedelta_dtype, is_datetimelike, is_datetimetz,
is_dtype_equal, is_extension_array_dtype, is_extension_type, is_float,
is_float_dtype, is_integer, is_integer_dtype, is_object_dtype, is_scalar,
is_string_dtype, is_timedelta64_dtype, is_timedelta64_ns_dtype,
is_unsigned_integer_dtype, pandas_dtype)
is_datetime_or_timedelta_dtype, is_datetimelike, is_dtype_equal,
is_extension_array_dtype, is_extension_type, is_float, is_float_dtype,
is_integer, is_integer_dtype, is_object_dtype, is_scalar, is_string_dtype,
is_timedelta64_dtype, is_timedelta64_ns_dtype, is_unsigned_integer_dtype,
pandas_dtype)
from .dtypes import (
DatetimeTZDtype, ExtensionDtype, PandasExtensionDtype, PeriodDtype)
from .generic import ABCDatetimeIndex, ABCPeriodIndex, ABCSeries
Expand Down Expand Up @@ -285,7 +285,7 @@ def maybe_promote(dtype, fill_value=np.nan):
fill_value = iNaT
else:
fill_value = iNaT
elif is_datetimetz(dtype):
elif is_datetime64tz_dtype(dtype):
if isna(fill_value):
fill_value = iNaT
elif is_extension_array_dtype(dtype) and isna(fill_value):
Expand Down Expand Up @@ -328,7 +328,7 @@ def maybe_promote(dtype, fill_value=np.nan):
# in case we have a string that looked like a number
if is_extension_array_dtype(dtype):
pass
elif is_datetimetz(dtype):
elif is_datetime64tz_dtype(dtype):
pass
elif issubclass(np.dtype(dtype).type, string_types):
dtype = np.object_
Expand Down Expand Up @@ -564,34 +564,6 @@ def invalidate_string_dtypes(dtype_set):
raise TypeError("string dtypes are not allowed, use 'object' instead")


def maybe_convert_string_to_object(values):
"""

Convert string-like and string-like array to convert object dtype.
This is to avoid numpy to handle the array as str dtype.
"""
if isinstance(values, string_types):
values = np.array([values], dtype=object)
elif (isinstance(values, np.ndarray) and
issubclass(values.dtype.type, (np.string_, np.unicode_))):
values = values.astype(object)
return values


def maybe_convert_scalar(values):
"""
Convert a python scalar to the appropriate numpy dtype if possible
This avoids numpy directly converting according to platform preferences
"""
if is_scalar(values):
dtype, values = infer_dtype_from_scalar(values)
try:
values = dtype(values)
except TypeError:
pass
return values


def coerce_indexer_dtype(indexer, categories):
""" coerce the indexer input array to the smallest dtype possible """
length = len(categories)
Expand Down Expand Up @@ -1206,7 +1178,7 @@ def construct_1d_arraylike_from_scalar(value, length, dtype):
np.ndarray / pandas type of length, filled with value

"""
if is_datetimetz(dtype):
if is_datetime64tz_dtype(dtype):
from pandas import DatetimeIndex
subarr = DatetimeIndex([value] * length, dtype=dtype)
elif is_categorical_dtype(dtype):
Expand Down
88 changes: 16 additions & 72 deletions pandas/core/dtypes/common.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
""" common type operations """
import warnings

import numpy as np

from pandas._libs import algos, lib
Expand Down Expand Up @@ -287,6 +289,8 @@ def is_datetimetz(arr):
Check whether an array-like is a datetime array-like with a timezone
component in its dtype.

.. deprecated:: 0.24.0

Parameters
----------
arr : array-like
Expand Down Expand Up @@ -320,8 +324,9 @@ def is_datetimetz(arr):
True
"""

# TODO: do we need this function?
# It seems like a repeat of is_datetime64tz_dtype.
warnings.warn("'is_datetimetz' is deprecated and will be removed in a "
"future version. Use 'is_datetime64tz_dtype' instead.",
FutureWarning, stacklevel=2)

return ((isinstance(arr, ABCDatetimeIndex) and
jbrockmendel marked this conversation as resolved.
Show resolved Hide resolved
getattr(arr, 'tz', None) is not None) or
Expand Down Expand Up @@ -363,6 +368,8 @@ def is_period(arr):
"""
Check whether an array-like is a periodical index.

.. deprecated:: 0.24.0

Parameters
----------
arr : array-like
Expand All @@ -382,8 +389,10 @@ def is_period(arr):
True
"""

# TODO: do we need this function?
# It seems like a repeat of is_period_arraylike.
warnings.warn("'is_period' is deprecated and will be removed in a future "
"version. Use 'is_period_dtype' or is_period_arraylike' "
"instead.", FutureWarning, stacklevel=2)

return isinstance(arr, ABCPeriodIndex) or is_period_arraylike(arr)
jbrockmendel marked this conversation as resolved.
Show resolved Hide resolved


Expand Down Expand Up @@ -743,8 +752,7 @@ def is_datetimelike(arr):

return (is_datetime64_dtype(arr) or is_datetime64tz_dtype(arr) or
is_timedelta64_dtype(arr) or
isinstance(arr, ABCPeriodIndex) or
is_datetimetz(arr))
isinstance(arr, ABCPeriodIndex))


def is_dtype_equal(source, target):
Expand Down Expand Up @@ -1050,54 +1058,6 @@ def is_int64_dtype(arr_or_dtype):
return issubclass(tipo, np.int64)


def is_int_or_datetime_dtype(arr_or_dtype):
"""
Check whether the provided array or dtype is of an
integer, timedelta64, or datetime64 dtype.

Parameters
----------
arr_or_dtype : array-like
The array or dtype to check.

Returns
-------
boolean : Whether or not the array or dtype is of an
integer, timedelta64, or datetime64 dtype.

Examples
--------
>>> is_int_or_datetime_dtype(str)
False
>>> is_int_or_datetime_dtype(int)
True
>>> is_int_or_datetime_dtype(float)
False
>>> is_int_or_datetime_dtype(np.uint64)
True
>>> is_int_or_datetime_dtype(np.datetime64)
True
>>> is_int_or_datetime_dtype(np.timedelta64)
True
>>> is_int_or_datetime_dtype(np.array(['a', 'b']))
False
>>> is_int_or_datetime_dtype(pd.Series([1, 2]))
True
>>> is_int_or_datetime_dtype(np.array([], dtype=np.timedelta64))
True
>>> is_int_or_datetime_dtype(np.array([], dtype=np.datetime64))
True
>>> is_int_or_datetime_dtype(pd.Index([1, 2.])) # float
False
"""

if arr_or_dtype is None:
return False
tipo = _get_dtype_type(arr_or_dtype)
return (issubclass(tipo, np.integer) or
issubclass(tipo, (np.datetime64, np.timedelta64)))


def is_datetime64_any_dtype(arr_or_dtype):
"""
Check whether the provided array or dtype is of the datetime64 dtype.
Expand Down Expand Up @@ -1619,22 +1579,6 @@ def is_float_dtype(arr_or_dtype):
return issubclass(tipo, np.floating)


def is_floating_dtype(arr_or_dtype):
"""Check whether the provided array or dtype is an instance of
numpy's float dtype.

.. deprecated:: 0.20.0

Unlike, `is_float_dtype`, this check is a lot stricter, as it requires
`isinstance` of `np.floating` and not `issubclass`.
"""

if arr_or_dtype is None:
return False
tipo = _get_dtype_type(arr_or_dtype)
return isinstance(tipo, np.floating)


def is_bool_dtype(arr_or_dtype):
"""
Check whether the provided array or dtype is of a boolean dtype.
Expand Down Expand Up @@ -1758,7 +1702,7 @@ def is_extension_type(arr):
return True
elif is_sparse(arr):
return True
elif is_datetimetz(arr):
elif is_datetime64tz_dtype(arr):
return True
return False

Expand Down Expand Up @@ -1991,7 +1935,7 @@ def _get_dtype_from_object(dtype):
return dtype
elif is_categorical(dtype):
return CategoricalDtype().type
elif is_datetimetz(dtype):
elif is_datetime64tz_dtype(dtype):
return DatetimeTZDtype(dtype).type
elif isinstance(dtype, np.dtype): # dtype object
try:
Expand Down
4 changes: 2 additions & 2 deletions pandas/core/dtypes/concat.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

from pandas.core.dtypes.common import (
_NS_DTYPE, _TD_DTYPE, is_bool_dtype, is_categorical_dtype,
is_datetime64_dtype, is_datetimetz, is_dtype_equal,
is_datetime64_dtype, is_datetime64tz_dtype, is_dtype_equal,
is_extension_array_dtype, is_interval_dtype, is_object_dtype,
is_period_dtype, is_sparse, is_timedelta64_dtype)
from pandas.core.dtypes.generic import (
Expand Down Expand Up @@ -39,7 +39,7 @@ def get_dtype_kinds(l):
typ = 'sparse'
elif isinstance(arr, ABCRangeIndex):
typ = 'range'
elif is_datetimetz(arr):
elif is_datetime64tz_dtype(arr):
# if to_concat contains different tz,
# the result must be object dtype
typ = str(arr.dtype)
Expand Down
5 changes: 3 additions & 2 deletions pandas/core/frame.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@
is_object_dtype,
is_extension_type,
is_extension_array_dtype,
is_datetimetz,
is_datetime64tz_dtype,
is_datetime64_any_dtype,
is_bool_dtype,
is_integer_dtype,
Expand Down Expand Up @@ -541,7 +541,8 @@ def _get_axes(N, K, index=index, columns=columns):
index, columns = _get_axes(len(values), 1)
return _arrays_to_mgr([values], columns, index, columns,
dtype=dtype)
elif (is_datetimetz(values) or is_extension_array_dtype(values)):
elif (is_datetime64tz_dtype(values) or
is_extension_array_dtype(values)):
# GH19157
if columns is None:
columns = [0]
Expand Down
4 changes: 2 additions & 2 deletions pandas/core/indexes/datetimes.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

from pandas.core.dtypes.common import (
_INT64_DTYPE, _NS_DTYPE, ensure_int64, is_datetime64_dtype,
is_datetime64_ns_dtype, is_datetimetz, is_dtype_equal, is_float,
is_datetime64_ns_dtype, is_datetime64tz_dtype, is_dtype_equal, is_float,
is_integer, is_integer_dtype, is_list_like, is_period_dtype, is_scalar,
is_string_like, pandas_dtype)
import pandas.core.dtypes.concat as _concat
Expand Down Expand Up @@ -259,7 +259,7 @@ def __new__(cls, data=None,
data = data._values

# data must be Index or np.ndarray here
if not (is_datetime64_dtype(data) or is_datetimetz(data) or
if not (is_datetime64_dtype(data) or is_datetime64tz_dtype(data) or
is_integer_dtype(data) or lib.infer_dtype(data) == 'integer'):
data = tools.to_datetime(data, dayfirst=dayfirst,
yearfirst=yearfirst)
Expand Down
Loading