Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sort broken using MultiIndex with mixed ascending #16934

Closed
alubbock opened this issue Jul 14, 2017 · 3 comments · Fixed by #16937
Closed

Sort broken using MultiIndex with mixed ascending #16934

alubbock opened this issue Jul 14, 2017 · 3 comments · Fixed by #16937
Labels
Bug MultiIndex Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Milestone

Comments

@alubbock
Copy link
Contributor

alubbock commented Jul 14, 2017

Code Sample, a copy-pastable example if possible

# Imports
import pandas as pd

# Set up a Series with a three level MultiIndex
arrays = [['bar', 'bar', 'baz', 'baz', 'foo', 'foo', 'qux', 'qux'],
  ['one', 'two', 'one', 'two', 'one', 'two', 'one', 'two'],
  [4, 3, 2, 1, 4, 3, 2, 1]]
tuples = list(zip(*arrays))
index = pd.MultiIndex.from_tuples(tuples, names=['first', 'second', 'third'])
s = pd.Series(range(8), index=index)

# Sort descending works
s.sort_index(level=['third', 'first'], ascending=False)

# But sort descending on third level, ascending on first does not
s.sort_index(level=['third', 'first'], ascending=[False, True])

Problem description

Sorting using a list for the ascending argument appears to have been implemented in #10905 but doesn't work in the above scenario. I get the following output:

>>> s.sort_index(level=['third', 'first'], ascending=[False, True])

first  second  third
qux    one     2        6
       two     1        7
foo    one     4        4
       two     3        5
baz    one     2        2
       two     1        3
bar    one     4        0
       two     3        1
dtype: int64

Expected Output

DataFrame sorted by index third descending then index first ascending:

>>> s.sort_index(level=['third', 'first'], ascending=[False, True])

first  second  third
bar    one     4        0
foo    one     4        4
bar    two     3        1
foo    two     3        5
baz    one     2        2
qux    one     2        6
baz    two     1        3
qux    two     1        7

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.0.final.0
python-bits: 64
OS: Darwin
OS-release: 16.6.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.20.2
pytest: None
pip: 9.0.1
setuptools: 35.0.1
Cython: None
numpy: 1.13.0
scipy: 0.19.0
xarray: None
IPython: 6.1.0
sphinx: None
patsy: None
dateutil: 2.6.0
pytz: 2017.2
blosc: None
bottleneck: None
tables: 3.4.2
numexpr: 2.6.2
feather: None
matplotlib: 2.0.0
openpyxl: None
xlrd: 1.0.0
xlwt: None
xlsxwriter: 0.9.6
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: 2.7.1 (dt dec pq3 ext lo64)
jinja2: None
s3fs: None
pandas_gbq: None
pandas_datareader: None

@gfyoung
Copy link
Member

gfyoung commented Jul 15, 2017

@alubbock : Would you mind adding (with a hand-constructed DataFrame) what you expect the output to look like? That would be useful in addition to your description.

Otherwise, this definitely looks like a bug. Nice catch! PR is welcome.

cc @roblevy

@alubbock
Copy link
Contributor Author

@gfyoung I've edited the issue to add expected output and removed the np.random call to make the example deterministic. Thanks.

@gfyoung
Copy link
Member

gfyoung commented Jul 15, 2017

@alubbock : Thanks! Feel free to submit a PR for this if you know where the issue is.

alubbock added a commit to alubbock/pandas that referenced this issue Jul 15, 2017
MultiIndex sorting with `sort_index` would fail when the `ascending`
argument was specified as a list but not all levels of the index were
specified in the `level` argument, or the levels were specified in
a different order to the MultiIndex.

This PR rectifies the issue and introduces a unit test based on pandas-dev#16934

Fixes: pandas-dev#16934
alubbock added a commit to alubbock/pandas that referenced this issue Jul 15, 2017
MultiIndex sorting with `sort_index` would fail when the `ascending`
argument was specified as a list but not all levels of the index were
specified in the `level` argument, or the levels were specified in
a different order to the MultiIndex.

This PR rectifies the issue and introduces a unit test based on pandas-dev#16934

Fixes: pandas-dev#16934
alubbock added a commit to alubbock/pandas that referenced this issue Jul 15, 2017
MultiIndex sorting with `sort_index` would fail when the `ascending`
argument was specified as a list but not all levels of the index were
specified in the `level` argument, or the levels were specified in
a different order to the MultiIndex.

This PR rectifies the issue and introduces a unit test based on pandas-dev#16934

Fixes: pandas-dev#16934
@jreback jreback added this to the 0.21.0 milestone Jul 15, 2017
@jreback jreback added the Reshaping Concat, Merge/Join, Stack/Unstack, Explode label Jul 15, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug MultiIndex Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants