Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Exception to the boolean frame comparison guarantee #16155

Closed
ResidentMario opened this issue Apr 26, 2017 · 3 comments
Closed

BUG: Exception to the boolean frame comparison guarantee #16155

ResidentMario opened this issue Apr 26, 2017 · 3 comments

Comments

@ResidentMario
Copy link
Contributor

ResidentMario commented Apr 26, 2017

The implementation for core._com_method_FRAME makes the following note:

            # straight boolean comparisions we want to allow all columns
            # (regardless of dtype to pass thru) See #4537 for discussion.

However, I think I stumbled on an exception here.

The root cause is that astype(bool) has a "hole" in it in the case of datetime dtypes. The following:

import pandas as pd
from io import StringIO

# create our surprisingly slippery frame
res = pd.read_csv(StringIO(""",dates1,dates2
0,1970-01-01 00:00:00.000000000,1970-01-01 00:00:00.000000000
1,1970-01-01 00:00:00.000000000,"""), index_col=0)
res['dates1'] = pd.to_datetime(res['dates1'])

res.astype(bool)

Raises:

TypeError: cannot astype a datetimelike from [datetime64[ns]] to [bool]

If you get this frame as a fillna output (not possible on master, but possible in the feature branch I am working on), it will raise this same error when passed through _com_method_FRAME, due to:

return res.fillna(True).astype(bool)

This is a problem in PR#15653.

@ResidentMario ResidentMario changed the title [BUG] Exception to the boolean frame comparison guarantee BUG: Exception to the boolean frame comparison guarantee Apr 26, 2017
@ResidentMario
Copy link
Contributor Author

xref #4537

@ResidentMario
Copy link
Contributor Author

The obvious solution would be to forcefully upcast any datetime columns to object dtype beforehand.

Highly inelegant, but perhaps necessary.

@ResidentMario
Copy link
Contributor Author

(closing because I don't think this can happen in master, just wanted to create something I can reference later)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant