Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merging on int / float columns with NaNs gives spurious warning #20998

Closed
jorisvandenbossche opened this issue May 10, 2018 · 0 comments
Closed
Labels
Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Milestone

Comments

@jorisvandenbossche
Copy link
Member

jorisvandenbossche commented May 10, 2018

In 0.23 we added a warning when trying to merge on int / float columns where the float values are not actually integers (#18352). No objections to that, but I think we should special case for NaNs, as this is a typical case that turns an integer column into floats, but does give the warning:

In [8]: df1 = pd.DataFrame({'col1': [0, 1], 'key': [1, 2]})
   ...: df2 = pd.DataFrame({'col2': ['a', 'b'], 'key': [2, np.nan]})

In [9]: pd.merge(df1, df2, on='key')
/home/joris/scipy/pandas/pandas/core/reshape/merge.py:969: UserWarning: You are merging on int and float columns where the float values are not equal to their int representation
  'representation', UserWarning)
Out[9]: 
   col1  key col2
0     1    2    a

So I think the above should not generate the warning.
A fix is probably to just mask for missing values in the check (PR coming).

@jorisvandenbossche jorisvandenbossche added the Reshaping Concat, Merge/Join, Stack/Unstack, Explode label May 10, 2018
@jorisvandenbossche jorisvandenbossche added this to the 0.23.0 milestone May 10, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

No branches or pull requests

1 participant