Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEPR: Series ndarray properties (strides, data, base, itemsize, flags) #20721

Merged
3 changes: 3 additions & 0 deletions doc/source/whatsnew/v0.23.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -877,6 +877,9 @@ Deprecations
- The ``convert_datetime64`` parameter in :func:`DataFrame.to_records` has been deprecated and will be removed in a future version. The NumPy bug motivating this parameter has been resolved. The default value for this parameter has also changed from ``True`` to ``None`` (:issue:`18160`).
- :func:`Series.rolling().apply() <pandas.core.window.Rolling.apply>`, :func:`DataFrame.rolling().apply() <pandas.core.window.Rolling.apply>`,
:func:`Series.expanding().apply() <pandas.core.window.Expanding.apply>`, and :func:`DataFrame.expanding().apply() <pandas.core.window.Expanding.apply>` have deprecated passing an ``np.array`` by default. One will need to pass the new ``raw`` parameter to be explicit about what is passed (:issue:`20584`)
- The ``data``, ``base``, ``strides``, ``flags`` and ``itemsize`` properties
of the ``Series`` and ``Index`` classes have been deprecated and will be
removed in a future version (:issue:`20419`).

.. _whatsnew_0230.prior_deprecations:

Expand Down
15 changes: 15 additions & 0 deletions pandas/core/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -737,11 +737,17 @@ def item(self):
@property
def data(self):
""" return the data pointer of the underlying data """
warnings.warn("{obj}.data is deprecated and will be removed "
"in a future version".format(obj=type(self).__name__),
FutureWarning, stacklevel=2)
return self.values.data

@property
def itemsize(self):
""" return the size of the dtype of the item of the underlying data """
warnings.warn("{obj}.itemsize is deprecated and will be removed "
"in a future version".format(obj=type(self).__name__),
FutureWarning, stacklevel=2)
return self._ndarray_values.itemsize

@property
Expand All @@ -752,6 +758,9 @@ def nbytes(self):
@property
def strides(self):
""" return the strides of the underlying data """
warnings.warn("{obj}.strudes is deprecated and will be removed "
"in a future version".format(obj=type(self).__name__),
FutureWarning, stacklevel=2)
return self._ndarray_values.strides

@property
Expand All @@ -762,13 +771,19 @@ def size(self):
@property
def flags(self):
""" return the ndarray.flags for the underlying data """
warnings.warn("{obj}.flags is deprecated and will be removed "
"in a future version".format(obj=type(self).__name__),
FutureWarning, stacklevel=2)
return self.values.flags

@property
def base(self):
""" return the base object if the memory of the underlying data is
shared
"""
warnings.warn("{obj}.base is deprecated and will be removed "
"in a future version".format(obj=type(self).__name__),
FutureWarning, stacklevel=2)
return self.values.base

@property
Expand Down
10 changes: 10 additions & 0 deletions pandas/core/indexes/datetimelike.py
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,16 @@ def ceil(self, freq):
class DatetimeIndexOpsMixin(object):
""" common ops mixin to support a unified interface datetimelike Index """

@property
def base(self):
""" return the base object if the memory of the underlying data is
shared
"""
# override deprecated property in IndexOpsMixin, as we still need
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

your explanation doesn't make sense, we are deprecating these no? you need to change the way its accessed in the code to remove the warnings.

Copy link
Member Author

@jorisvandenbossche jorisvandenbossche Apr 19, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The explanation perfectly makes sense, but you don't like the way I solved it I suppose (I agree it is not the nicest, but I was thinking that this would only be temporarily until DatetimeTZBlock is an ExtensionBlock).

To fix the usage itself, .base is used in two placed:

  • Block.is_view: this I can override in DatetimeTzBlock to check self.values.values.base
  • concatenate_join_units:
    if copy and concat_values.base is not None:
    Not fully sure about this one what to do here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my point is this method should not exist (as you are ddeprecateding), and would rather have you fix the usage

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I said, I agree with that. Do you have a suggestion for the second case? (inside concatenate_join_units) In #20745 I need to touch the same code, and checked if the values have a base attribute, but that also feels hacky

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, see my comment there, only check base if its an ndarray

# this for internals (DatetimeIndex/TimedeltaIndex is stored as
# values in Blocks)
return self.values.base

def equals(self, other):
"""
Determines if two Index objects contain the same elements.
Expand Down
2 changes: 1 addition & 1 deletion pandas/tests/groupby/aggregate/test_other.py
Original file line number Diff line number Diff line change
Expand Up @@ -328,7 +328,7 @@ def test_series_agg_multi_pure_python():
'F': np.random.randn(11)})

def bad(x):
assert (len(x.base) > 0)
assert (len(x.values.base) > 0)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really sure what the purpose of this assert actually is (was introduced in 71e9046)

return 'foo'

result = data.groupby(['A', 'B']).agg(bad)
Expand Down
2 changes: 1 addition & 1 deletion pandas/tests/groupby/test_groupby.py
Original file line number Diff line number Diff line change
Expand Up @@ -1618,7 +1618,7 @@ def convert_fast(x):

def convert_force_pure(x):
# base will be length 0
assert (len(x.base) > 0)
assert (len(x.values.base) > 0)
return Decimal(str(x.mean()))

grouped = s.groupby(labels)
Expand Down
2 changes: 1 addition & 1 deletion pandas/tests/indexes/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
class Base(object):
""" base class for index sub-class tests """
_holder = None
_compat_props = ['shape', 'ndim', 'size', 'itemsize', 'nbytes']
_compat_props = ['shape', 'ndim', 'size', 'nbytes']

def setup_indices(self):
for name, idx in self.indices.items():
Expand Down
17 changes: 13 additions & 4 deletions pandas/tests/test_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -316,16 +316,25 @@ def test_ndarray_compat_properties(self):

for o in self.objs:
# Check that we work.
for p in ['shape', 'dtype', 'flags', 'T',
'strides', 'itemsize', 'nbytes']:
for p in ['shape', 'dtype', 'T', 'nbytes']:
assert getattr(o, p, None) is not None

assert hasattr(o, 'base')
# deprecated properties
for p in ['flags', 'strides', 'itemsize']:
with tm.assert_produces_warning(FutureWarning):
assert getattr(o, p, None) is not None

# not deprecated for datetime-like indices because they are used
# inside blocks
if not isinstance(o, (DatetimeIndex, TimedeltaIndex, PeriodIndex)):
with tm.assert_produces_warning(FutureWarning):
assert hasattr(o, 'base')

# If we have a datetime-like dtype then needs a view to work
# but the user is responsible for that
try:
assert o.data is not None
with tm.assert_produces_warning(FutureWarning):
assert o.data is not None
except ValueError:
pass

Expand Down