DEPR: DataFrame.get_dtype_counts #27145

mroeschke · 2019-06-30T16:59:36Z

xref DEPR: let's deprecate #18262
tests added / passed
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

Unfortunately, .dtype.value_counts() isn't a perfect replacement for get_value_counts() (IMO it's better), so the test replacements calls the internal representation:

In [2]: df = pd.DataFrame([[1, 1.5, pd.Timestamp.now(), True]])

In [3]: df
Out[3]:
   0    1                          2     3
0  1  1.5 2019-06-30 11:55:24.380126  True

In [7]: df.get_dtype_counts().index
Out[7]: Index(['float64', 'int64', 'datetime64[ns]', 'bool'], dtype='object')

In [8]: df.dtypes.value_counts().index
Out[8]: Index([datetime64[ns], bool, float64, int64], dtype='object')

pep8speaks · 2019-06-30T22:27:04Z

Hello @mroeschke! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2019-07-03 02:19:59 UTC

jreback · 2019-06-30T22:31:11Z

pandas/tests/frame/test_constructors.py

@@ -1693,7 +1693,7 @@ def test_constructor_datetimes_with_nulls(self):
        for arr in [np.array([None, None, None, None,
                              datetime.now(), None]),
                    np.array([None, None, datetime.now(), None])]:
-            result = DataFrame(arr).get_dtype_counts()
+            result = Series(DataFrame(arr)._data.get_dtype_counts())


don’t move it to a private method

use .dtypes.value_counts()

jreback · 2019-07-01T12:01:13Z

pandas/core/computation/expressions.py

@@ -79,11 +79,11 @@ def _can_use_numexpr(op, op_str, a, b, dtype_check):
            # check for dtype compatibility
            dtypes = set()
            for o in [a, b]:
-                if hasattr(o, 'get_dtype_counts'):
-                    s = o.get_dtype_counts()
+                if hasattr(o, '_data'):


can prob just do
o.dtypes.value_counts() (may need a hasattr)

jreback · 2019-07-01T12:01:36Z

pandas/tests/frame/test_api.py

@@ -433,9 +433,10 @@ def test_with_datetimelikes(self):
                         'B': timedelta_range('1 day', periods=10)})
        t = df.T

-        result = t.get_dtype_counts()
+        #result = Series(t._data.get_dtype_counts())


comment here

jreback · 2019-07-01T12:01:45Z

pandas/core/generic.py

@@ -5290,6 +5290,9 @@ def get_dtype_counts(self):
        object     1
        dtype: int64
        """
+        warnings.warn("`get_dtype_counts` has been deprecated and will be "


can you update the docstring and add deprecated

Can we recommend .dtypes.value_counts() here instead? Or... we're in generic.py so that may be too hard?

Yeah unfortunately that solution does not work for Series, but I could add for DataFrames use .dtypes.value_counts()

sure, just need something as a replacement (may also want to add in the doc-string itself)

…pe_counts

mroeschke · 2019-07-02T17:40:11Z

All green.

jreback

1 small change & doc-string updates, otherwise lgtm.

jreback · 2019-07-02T19:06:51Z

pandas/core/frame.py

@@ -2326,7 +2326,7 @@ def _sizeof_fmt(num, size_qualifier):
            else:
                _verbose_repr()

-        counts = self.get_dtype_counts()
+        counts = self._data.get_dtype_counts()


I think this is okay. It's internal usage and slightly more performant I would think than dtype.value_counts() (left as a dictionary as opposed to constructing the Series)

can you remove get_dtype_counts() from blocks its unecessary as well

Looks to be needed to get the dtypes later on for info?

jreback · 2019-07-02T19:07:24Z

pandas/core/generic.py

@@ -5290,6 +5290,9 @@ def get_dtype_counts(self):
        object     1
        dtype: int64
        """
+        warnings.warn("`get_dtype_counts` has been deprecated and will be "


sure, just need something as a replacement (may also want to add in the doc-string itself)

jreback · 2019-07-03T11:27:39Z

thanks @mroeschke

Matt Roeschke added 3 commits June 30, 2019 11:18

DEPR: get_dtype_values

2fe603d

Add test and whatsnew note:

735a2fd

Flake8

8473885

mroeschke added the Deprecate Functionality to remove in pandas label Jun 30, 2019

Matt Roeschke added 2 commits June 30, 2019 15:30

Start converting tests to compare np dtypes

faeb972

keys --> keys(), first pass at converting tests

95a7075

jreback requested changes Jun 30, 2019

View reviewed changes

Matt Roeschke added 8 commits June 30, 2019 20:29

Change more tests

eb5213c

Change more tests

06af7fc

More progress

879485b

convert more tests

0c82f16

Converting

525fe51

Convert more tests

592659f

convert tests

0df4dd9

flake8

eba5396

jreback requested changes Jul 1, 2019

View reviewed changes

jreback added this to the 0.25.0 milestone Jul 1, 2019

jreback mentioned this pull request Jul 1, 2019

DEPR: let's deprecate #18262

Closed

34 tasks

jsexauer mentioned this pull request Jul 1, 2019

DEPR: Clean up list of deprecations from prior versions #6581

Closed

1 task

Matt Roeschke added 10 commits July 1, 2019 09:15

Merge remote-tracking branch 'upstream/master' into deprecate_get_dty…

eeba8a4

…pe_counts

Merge remote-tracking branch 'upstream/master' into deprecate_get_dty…

4419114

…pe_counts

update docstring

4ce1194

Address some failures

cb6c8bd

Merge remote-tracking branch 'upstream/master' into deprecate_get_dty…

b3b839a

…pe_counts

typo

abe310c

Merge remote-tracking branch 'upstream/master' into deprecate_get_dty…

aa172d6

…pe_counts

isort

6cb5d13

Fix doc warnings

ca70a46

Edit tests for dict sorting and platform ints

86c0a08

Merge remote-tracking branch 'upstream/master' into deprecate_get_dty…

523ab60

…pe_counts

Add alternative method

852bd64

jreback requested changes Jul 2, 2019

View reviewed changes

Matt Roeschke and others added 2 commits July 2, 2019 13:55

Add doc note

e700c63

Merge branch 'master' into PR_TOOL_MERGE_PR_27145

475e361

jreback approved these changes Jul 3, 2019

View reviewed changes

jreback merged commit 54c0d5a into pandas-dev:master Jul 3, 2019

mroeschke deleted the deprecate_get_dtype_counts branch July 3, 2019 15:13

ccharlesgb mentioned this pull request Jul 29, 2019

Operators between DataFrame and Series fail on large dataframes #27636

Closed

ccharlesgb mentioned this pull request Aug 6, 2019

BUG: _can_use_numexpr fails when passed large Series #27773

Merged

5 tasks

jreback mentioned this pull request Dec 1, 2019

DEPR: deprecations log for removed issues #13777

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DEPR: DataFrame.get_dtype_counts #27145

DEPR: DataFrame.get_dtype_counts #27145

mroeschke commented Jun 30, 2019 •

edited

Loading

pep8speaks commented Jun 30, 2019 •

edited

Loading

jreback Jun 30, 2019

jreback Jul 1, 2019

jreback Jul 1, 2019

jreback Jul 1, 2019

TomAugspurger Jul 2, 2019

mroeschke Jul 2, 2019

jreback Jul 2, 2019

mroeschke commented Jul 2, 2019

jreback left a comment

jreback Jul 2, 2019

mroeschke Jul 2, 2019

jreback Jul 2, 2019

mroeschke Jul 2, 2019

jreback Jul 2, 2019

jreback commented Jul 3, 2019

DEPR: DataFrame.get_dtype_counts #27145

DEPR: DataFrame.get_dtype_counts #27145

Conversation

mroeschke commented Jun 30, 2019 • edited Loading

pep8speaks commented Jun 30, 2019 • edited Loading

Comment last updated at 2019-07-03 02:19:59 UTC

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mroeschke commented Jul 2, 2019

jreback left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Jul 3, 2019

mroeschke commented Jun 30, 2019 •

edited

Loading

pep8speaks commented Jun 30, 2019 •

edited

Loading