Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Series.__sub__ non-nano datetime64 #18783

Merged
merged 13 commits into from
Dec 28, 2017
Merged

Conversation

jbrockmendel
Copy link
Member

The original bug report was for Series. This fixes that bug and includes a test that checks for DatetimeIndex while we're at it. I checked and this does not fix the analogous problem in `DataFrame. I'm hoping someone else will pick up the torch on that b/c the broadcast/dispatch is still something of a mystery to me.

@@ -671,7 +676,7 @@ def _arith_method_SERIES(op, name, str_rep, fill_zeros=None, default_axis=None,
"""
def na_op(x, y):
import pandas.core.computation.expressions as expressions

#
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pls don't add extraneous things

res = dt64 - ser
tm.assert_series_equal(res, -expected)

# check for DatetimeIndex and DataFrame while we're at it
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make the comment relevant

res = dt64 - dti
tm.assert_index_equal(res, pd.Index(-expected))

# TODO: This is still broken for ser.to_frame()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

then if you want to add a test with an xfail would be helpful. (and an issue reference), TODO's are only so good.

@@ -28,6 +28,31 @@
from .common import TestData


class TestDatetimeLikeArithmetic(object):
def test_sub_datetime64_not_ns(self):
# GH#7996
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parameterize this

@jreback jreback added Compat pandas objects compatability with Numpy or Python functions Datetime Datetime data dtype labels Dec 14, 2017
@codecov
Copy link

codecov bot commented Dec 14, 2017

Codecov Report

Merging #18783 into master will increase coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #18783      +/-   ##
==========================================
+ Coverage   91.59%   91.59%   +<.01%     
==========================================
  Files         150      150              
  Lines       48959    48961       +2     
==========================================
+ Hits        44843    44847       +4     
+ Misses       4116     4114       -2
Flag Coverage Δ
#multiple 89.96% <100%> (ø) ⬆️
#single 41.13% <50%> (ø) ⬆️
Impacted Files Coverage Δ
pandas/core/ops.py 90.2% <100%> (+0.03%) ⬆️
pandas/util/testing.py 84.9% <0%> (+0.21%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cdebcf3...0dbe62c. Read the comment docs.

@jbrockmendel
Copy link
Member Author

Timeout on circleCI in collection, test_geopandas in travis. Will wait to re-push until given the go-ahead.

@jreback
Copy link
Contributor

jreback commented Dec 15, 2017

yeah don't re-push anything.

def test_sub_datetime64_not_ns(self):
# GH#7996
ser = Series(date_range('20130101', periods=3))
dt64 = np.datetime64('2013-01-01')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test with a tz-aware as well (as I think that may hit your path, unless its getting caught earlier)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It gets caught earlier: tz compat raises ValueError in ops._validate

tm.assert_index_equal(res, pd.Index(-expected))

@pytest.mark.xfail(reason='GH#7996 datetime64 units not converted to nano')
def test_frame_sub_datetime64_not_ns(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we have a separate issue for this? if not let's create one and reference it (as closing #7996 with this PR)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just opened #18874.

tm.assert_series_equal(res, expected)

res = dt64 - ser
tm.assert_series_equal(res, -expected)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you parametrize

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not without introducing a whole lot of extra boilerplate.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure that is the case. you are repeating Series/DTI testing.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, just changed. It is much less obvious to a casual reader exactly what this is testing.

tm.assert_series_equal(res, expected)

res = dt64 - ser
tm.assert_series_equal(res, -expected)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure that is the case. you are repeating Series/DTI testing.

Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small change. ping on green.

@@ -962,6 +962,34 @@ def test_timedelta64_ops_nat(self):


class TestDatetimeSeriesArithmetic(object):
@pytest.mark.parametrize('box_cls, assert_func', [(Series,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this formatting is a bit atypical, put a more like

@pytest.mark.parametrize(
    'box, assert_func', 
    [(Series, tm.assert_series_equal),
      (pd.Index, tm.assert_index_equal)])

is much better, also rename box_cls -> box (consistent with elsewhere)

@jreback jreback added this to the 0.23.0 milestone Dec 23, 2017
@jbrockmendel
Copy link
Member Author

Ping

@jbrockmendel
Copy link
Member Author

This somehow doesn't need a whatsnew merge

@jreback jreback merged commit ef75390 into pandas-dev:master Dec 28, 2017
@jreback
Copy link
Contributor

jreback commented Dec 28, 2017

thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Compat pandas objects compatability with Numpy or Python functions Datetime Datetime data dtype
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: coercion of non-M8[ns] in datetime ops
2 participants