Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG/ERR: wrong error in DataFrame.drop with non-unique datetime index + invalid keys #30399

Closed
jorisvandenbossche opened this issue Dec 22, 2019 · 1 comment · Fixed by #30446
Labels
Bug Error Reporting Incorrect or improved errors from pandas Indexing Related to indexing on series/frames, not to indexes themselves
Milestone

Comments

@jorisvandenbossche
Copy link
Member

Consider this example, where there is a DataFrame with a non-unique DatetimeIndex:

In [8]: df = pd.DataFrame(np.random.randn(5, 3), columns=['a', 'b', 'c'], index=pd.date_range("2012", freq='H', periods=5))

In [9]: df = df.iloc[[0, 2, 2, 3]] 

In [10]: df
Out[10]: 
                            a         b         c
2012-01-01 00:00:00 -1.534726 -0.559295  0.207194
2012-01-01 02:00:00 -1.072027  0.376595  0.407512
2012-01-01 02:00:00 -1.072027  0.376595  0.407512
2012-01-01 03:00:00  0.581614  1.782635 -0.678197

If you then use drop to drop some columns, but forget to specify columns= or axis=1 (so you are actually dropping rows), you get a wrong error and very confusing error message:

In [10]: df.drop(['a', 'b'])
...

~/scipy/pandas/pandas/core/indexes/base.py in get_indexer_non_unique(self, target)
   4559             tgt_values = target._ndarray_values
   4560 
-> 4561         indexer, missing = self._engine.get_indexer_non_unique(tgt_values)
   4562         return ensure_platform_int(indexer), missing
   4563 

~/scipy/pandas/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_indexer_non_unique()

TypeError: 'NoneType' object is not iterable

Tested with pandas 0.25 and pandas master.

@jorisvandenbossche jorisvandenbossche added Bug Indexing Related to indexing on series/frames, not to indexes themselves Error Reporting Incorrect or improved errors from pandas labels Dec 22, 2019
@jorisvandenbossche jorisvandenbossche added this to the Contributions Welcome milestone Dec 22, 2019
@fujiaxiang
Copy link
Member

Hi, I'm taking a look at this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Error Reporting Incorrect or improved errors from pandas Indexing Related to indexing on series/frames, not to indexes themselves
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants