ENH: Allow rename_axis to specify index and columns arguments #20046

Dr-Irv · 2018-03-07T23:21:13Z

closes API: Allow dictionary argument in rename_axis to change some names of MultiIndex #19978
tests added / passed
- test_rename_axis_mapper for DataFrame and Series
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

gfyoung · 2018-03-08T18:15:46Z

pandas/core/generic.py

-        """Alter the name of the index or columns.
+    def rename_axis(self, mapper=None, **kwargs):
+        """Alter the name of the index or name of index backing the
+        columns.


Let's keep this to be one line.

gfyoung · 2018-03-08T18:16:35Z

pandas/core/generic.py

@@ -749,6 +749,20 @@ def swaplevel(self, i=-2, j=-1, axis=0):
    # ----------------------------------------------------------------------
    # Rename

+    # renamer function if passed a dict


I know you just moved the function, but let's take this opportunity to write a brief docstring.

codecov · 2018-03-09T00:43:00Z

Codecov Report

❗ No coverage uploaded for pull request base (master@0d6cb3c). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master   #20046   +/-   ##
=========================================
  Coverage          ?   92.18%           
=========================================
  Files             ?      161           
  Lines             ?    51184           
  Branches          ?        0           
=========================================
  Hits              ?    47185           
  Misses            ?     3999           
  Partials          ?        0

Flag	Coverage Δ
#multiple	`90.62% <100%> (?)`
#single	`42.23% <50%> (?)`

Impacted Files	Coverage Δ
pandas/core/generic.py	`96.81% <100%> (ø)`
pandas/core/common.py	`98.41% <100%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d6cb3c...1263a47. Read the comment docs.

jreback · 2018-03-09T00:53:18Z

this implementation needs to follow the coding standards that are present in .rename, pls use this as a guide.

you have text saying that this has 2 calling conventions. This alone is making me -1 on this whole idea.

jreback

see comments

Dr-Irv · 2018-03-09T02:31:13Z

@jreback I'm not sure which coding standards that you are referring to in .rename. Can you be more specific? Note that for .rename_axis, the signature has to be different in order to preserve the deprecated behavior.

Secondly, with respect to the 2 calling conventions, that is true for .rename as well. I just copied that documentation pattern from there. Look at the comments from @jorisvandenbossche in #19978 where he suggested adding the index and columns keyword to rename_axis, which, IMHO, is a good idea.

Dr-Irv · 2018-03-30T17:04:42Z

@jreback See above. Awaiting your feedback or maybe @jorisvandenbossche

jreback · 2018-07-08T16:07:44Z

@Dr-Irv if you can rebase can have a look

gfyoung

cc @jreback

jreback · 2018-09-25T16:44:51Z

@Dr-Irv can you rebase

pep8speaks · 2018-09-25T18:21:09Z

Hello @Dr-Irv! Thanks for updating the PR.

There are no PEP8 issues in the file pandas/core/generic.py !
There are no PEP8 issues in the file pandas/tests/frame/test_alter_axes.py !
There are no PEP8 issues in the file pandas/tests/io/test_pytables.py !
There are no PEP8 issues in the file pandas/tests/reshape/test_concat.py !
There are no PEP8 issues in the file pandas/tests/series/test_alter_axes.py !
There are no PEP8 issues in the file pandas/tests/test_panel.py !

Dr-Irv · 2018-09-26T02:26:57Z

@jreback Merged with latest master. Current failure in travis was due to a network error.

jreback · 2018-09-26T10:33:22Z

@TomAugspurger @jorisvandenbossche if you'd have a look.

jreback · 2018-09-26T10:34:08Z

doc/source/whatsnew/v0.24.0.txt

@@ -181,7 +181,8 @@ Other Enhancements
 - :func:`read_html` copies cell data across ``colspan`` and ``rowspan``, and it treats all-``th`` table rows as headers if ``header`` kwarg is not given and there is no ``thead`` (:issue:`17054`)
 - :meth:`Series.nlargest`, :meth:`Series.nsmallest`, :meth:`DataFrame.nlargest`, and :meth:`DataFrame.nsmallest` now accept the value ``"all"`` for the ``keep`` argument. This keeps all ties for the nth largest/smallest value (:issue:`16818`)
 - :class:`IntervalIndex` has gained the :meth:`~IntervalIndex.set_closed` method to change the existing ``closed`` value (:issue:`21670`)
- :func:`~DataFrame.to_csv`, :func:`~Series.to_csv`, :func:`~DataFrame.to_json`, and :func:`~Series.to_json` now support ``compression='infer'`` to infer compression based on filename extension (:issue:`15008`).
+- :func:`~DataFrame.to_csv` and :func:`~DataFrame.to_json` now support ``compression='infer'`` to infer compression based on filename (:issue:`15008`)


actually can you move this note to a sub-section and show the previous and new behavior. I now this is not a super common method, but its a fairly meaty change.

@jreback latest commit puts rename_axis() change in a subsection with an example. Also, I had messed up the merge of whatsnew in previous commit, so that is now fixed.

jreback · 2018-10-18T16:17:21Z

can you rebase

Dr-Irv · 2018-10-19T17:22:39Z

@jreback merged in latest master. All green

jreback · 2018-10-24T12:42:35Z

doc/source/basics.rst

@@ -1464,10 +1464,25 @@ for altering the ``Series.name`` attribute.

   s.rename("scalar-name")

+The Panel class has a related :meth:`~Panel.rename` class which can rename


this is not needed, we are not advertising Panel any longer

OK, I deleted that, but Panel appears a lot in that document!

i know, but no need to add at this point

jreback · 2018-10-24T12:44:00Z

pandas/core/generic.py

+    def _get_rename_function(self, mapper):
+        """
+        Returns a function that will map names/labels, dependent if mapper
+        is a dict, Series or just a function.


this doesn't need to be here (in the class). is it used in more than 1 place?

Yes, it is used in both rename() and rename_axis()

right, but it doesn't need self right? can you move to pandas/core/common.py i think is ok for this

I have moved it and removed leading underscore

jreback · 2018-10-24T12:44:16Z

pandas/core/generic.py

+
+            Use either ``mapper`` and ``axis`` to
+            specify the axis to target with ``mapper``, or ``index``
+            and/or ``columns``.


add a versionchanged

Dr-Irv · 2018-10-24T22:04:54Z

@jreback Changes made, and all green.

TomAugspurger

Haven't looked at the tests yet, but a few comments first.

TomAugspurger · 2018-10-25T12:10:42Z

pandas/core/generic.py

        """
-        Alter the name of the index or columns.
+        Alter the name of the index or name of index backing the columns.


I don't understand this change, is it a typo?

Oh, I think I saying "alter the name of the Index object that is the columns"?

If so, I think that index needs to be capitalized. But I think the older version was fine.

I will use "alter the name of the Index object that is the columns."

TomAugspurger · 2018-10-25T12:12:23Z

pandas/core/generic.py

+        * ``(mapper, axis={'index', 'columns'}, ...)``
+
+        The first calling convention will only modify the names of
+        the index and/or the names of the index backing the columns.


same: index -> Index, or just remove.

changed to "names of the Index object that is the columns."

pandas/core/generic.py

TomAugspurger · 2018-10-25T12:18:03Z

pandas/core/common.py

+    Returns a function that will map names/labels, dependent if mapper
+    is a dict, Series or just a function.
+    """
+    if isinstance(mapper, (dict, ABCSeries)):


using compat.Mapping instead of dict.

pandas/core/common.py

TomAugspurger · 2018-10-25T12:20:14Z

Could also maybe use a short docnote in advanced.rst.

Dr-Irv · 2018-10-25T18:58:40Z

I've added a section on renaming to advanced.rst.

Dr-Irv · 2018-10-25T22:20:43Z

@TomAugspurger I did the changes. See remark at #20046 (comment) . I didn't make a change there.

There was a failure in the CI due to a timeout on travis. Rather than resubmit, I figured you could review what's here and see if any other changes are needed.

jreback

lgtm. small comments. nice docs!

jreback · 2018-10-26T02:06:15Z

doc/source/basics.rst

+
+.. ipython:: python
+
+   df = pd.DataFrame({'x': [1,2,3,4,5,6], 'y': [10,20,30,40,50,60]},


can you make this pep-y, iow spaces after commas (you can also line up the x and y columns on separate lines)

doc/source/whatsnew/v0.24.0.txt

jreback · 2018-10-26T02:07:27Z

pandas/core/common.py

@@ -454,3 +454,21 @@ def _pipe(obj, func, *args, **kwargs):
        return func(*args, **kwargs)
    else:
        return func(obj, *args, **kwargs)
+
+
+def get_rename_function(mapper):


can you make this private (e.g. _get_rename_function)

jreback · 2018-10-26T02:08:12Z

pandas/tests/frame/test_alter_axes.py

+        mi = MultiIndex.from_product([['a', 'b', 'c'], [1, 2]],
+                                     names=['ll', 'nn'])
+        df = DataFrame({'x': [i for i in range(len(mi))],
+                        'y': [i * 10 for i in range(len(mi))]},


can you sprinkle some comments to delineate the cases you are testing

Dr-Irv · 2018-10-26T15:25:27Z

@jreback @TomAugspurger all green

TomAugspurger · 2018-10-26T20:20:05Z

pandas/core/generic.py

+                # and not a list or scalar, then call rename
+                msg = ("Using 'rename_axis' to alter labels is deprecated. "
+                       "Use '.rename' instead")
+                warnings.warn(msg, FutureWarning, stacklevel=2)


I think if you increase stack level by 1, you can revert your changes to the tests below. The stack level increased by 1 since you're using the decorator to fix the function signature.

Thanks for the tip. It worked.

TomAugspurger · 2018-10-26T20:20:29Z

pandas/tests/frame/test_alter_axes.py

@@ -522,22 +522,72 @@ def test_rename_axis_inplace(self, float_frame):
    def test_rename_axis_warns(self):
        # https://github.com/pandas-dev/pandas/issues/17833
        df = DataFrame({"A": [1, 2], "B": [1, 2]})
-        with tm.assert_produces_warning(FutureWarning) as w:
+        with tm.assert_produces_warning(FutureWarning,
+                                        check_stacklevel=False) as w:


So these check_stacklevel changes can hopefully all be reverted.

Dr-Irv · 2018-10-27T17:03:22Z

@jreback @TomAugspurger Almost all Green, but the failure was due to something on Azure "before install". Let me know if there are more changes, and if there is some way to get that job to run again.

jreback · 2018-10-28T03:29:11Z

@Dr-Irv ok lgtm. can you rebase, lots of isort things going on. ping on green.

Dr-Irv · 2018-10-29T15:18:13Z

@jreback merged with latest and all green

TomAugspurger · 2018-10-29T18:53:20Z

Thanks!

…y_tests * repo_org/master: (52 commits) ENH: Allow rename_axis to specify index and columns arguments (pandas-dev#20046) STY: proposed isort settings [ci skip] [skip ci] [ciskip] [skipci] (pandas-dev#23366) MAINT: Remove extraneous test.parquet file CLN: Follow-up comments to pandas-devgh-23392 (pandas-dev#23401) BUG GH23282 calling min on series of NaT returns NaT (pandas-dev#23289) unpin openpyxl (pandas-dev#23361) REF: collect ops dispatch functions in one place, try to de-duplicate SparseDataFrame methods (pandas-dev#23060) CLN: Remove pandas.tools module (pandas-dev#23376) CLN: Remove some dtype methods from API (pandas-dev#23390) CLN: Cleanup toplevel namespace shims (pandas-dev#23386) DOC: fixup whatsnew note for GH21394 (pandas-dev#23355) Fix import format at pandas/tests/extension directory (pandas-dev#23365) DOC: Remove Series.sortlevel from api.rst (pandas-dev#23395) API: Disallow dtypes w/o frequency when casting (pandas-dev#23392) BUG/TST/REF: Datetimelike Arithmetic Methods (pandas-dev#23215) STYLE: lint add np.nan* funcs to cython_table (pandas-dev#22109) Run Isort on tests/util single PR (pandas-dev#23347) BUG: Fix date_range overflow (pandas-dev#23345) Run Isort on tests/arrays single PR (pandas-dev#23346) ...

…-dev#20046)

Dr-Irv added 4 commits March 7, 2018 10:01

First version

bb12644

some doc changes

99011df

full implementation

4b45493

First test of new rename_axis

5ff65a6

gfyoung added Enhancement MultiIndex labels Mar 8, 2018

gfyoung reviewed Mar 8, 2018

View reviewed changes

Dr-Irv added 2 commits March 8, 2018 14:29

Merge remote-tracking branch 'upstream/master' into issue19978

36e5412

Doc changes and resolve conflicts with master

a14e6a1

Dr-Irv mentioned this pull request Mar 8, 2018

API: Allow dictionary argument in rename_axis to change some names of MultiIndex #19978

Closed

jreback requested changes Mar 9, 2018

View reviewed changes

Dr-Irv added 2 commits July 9, 2018 16:08

Merge in latest master

c8d6604

Fix doc merge

5806d9a

gfyoung approved these changes Jul 9, 2018

View reviewed changes

Merge to latest master

f28246f

fix flake8 issue

2f53685

jreback approved these changes Sep 26, 2018

View reviewed changes

jreback added this to the 0.24.0 milestone Sep 26, 2018

jreback requested changes Sep 26, 2018

View reviewed changes

Make whatsnew show example

5fd1ddf

Merge with master

5ec7748

jreback requested changes Oct 24, 2018

View reviewed changes

Dr-Irv added 2 commits October 24, 2018 10:21

Merge remote-tracking branch 'upstream/master' into issue19978

5c9c3ce

doc fix. Move get_rename_function to common

0542cf0

TomAugspurger reviewed Oct 25, 2018

View reviewed changes

Fix signature of rename_axis, add docs

39938a1

jreback requested changes Oct 26, 2018

View reviewed changes

Doc fixes, make get_rename_function private

65e3733

TomAugspurger reviewed Oct 26, 2018

View reviewed changes

stacklevel for warnings

e965c5b

jreback approved these changes Oct 28, 2018

View reviewed changes

merge to handle isort

1263a47

TomAugspurger merged commit a2e5994 into pandas-dev:master Oct 29, 2018

Dr-Irv deleted the issue19978 branch November 2, 2018 13:24

tm9k1 pushed a commit to tm9k1/pandas that referenced this pull request Nov 19, 2018

ENH: Allow rename_axis to specify index and columns arguments (pandas…

8fd0397

…-dev#20046)

Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019

ENH: Allow rename_axis to specify index and columns arguments (pandas…

9012100

…-dev#20046)

Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019

ENH: Allow rename_axis to specify index and columns arguments (pandas…

651803b

…-dev#20046)

		@@ -1464,10 +1464,25 @@ for altering the ``Series.name`` attribute.

		s.rename("scalar-name")

		The Panel class has a related :meth:`~Panel.rename` class which can rename


		.. ipython:: python

		df = pd.DataFrame({'x': [1,2,3,4,5,6], 'y': [10,20,30,40,50,60]},

ENH: Allow rename_axis to specify index and columns arguments #20046

ENH: Allow rename_axis to specify index and columns arguments #20046

Conversation

Dr-Irv commented Mar 7, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Mar 9, 2018 • edited Loading

Codecov Report

jreback commented Mar 9, 2018

jreback left a comment

Choose a reason for hiding this comment

Dr-Irv commented Mar 9, 2018

Dr-Irv commented Mar 30, 2018

jreback commented Jul 8, 2018

gfyoung left a comment

Choose a reason for hiding this comment

jreback commented Sep 25, 2018

pep8speaks commented Sep 25, 2018

Dr-Irv commented Sep 26, 2018

jreback commented Sep 26, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Oct 18, 2018

Dr-Irv commented Oct 19, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Dr-Irv commented Oct 24, 2018

TomAugspurger left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomAugspurger commented Oct 25, 2018

Dr-Irv commented Oct 25, 2018

Dr-Irv commented Oct 25, 2018

jreback left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Dr-Irv commented Oct 26, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Dr-Irv commented Oct 27, 2018

jreback commented Oct 28, 2018

Dr-Irv commented Oct 29, 2018

TomAugspurger commented Oct 29, 2018

codecov bot commented Mar 9, 2018 •

edited

Loading