Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed Issue Preventing Agg on RollingGroupBy Objects #21323

Merged
merged 12 commits into from
Sep 26, 2018

Conversation

WillAyd
Copy link
Member

@WillAyd WillAyd commented Jun 5, 2018

AFAICT the fact that RollingGroupBy could not use agg with a list of functions is simply due to the fact that the GroupByMixing it inherits from could not handle the reduction in dimensions that occurs via the normal aggregation functions.

@WillAyd WillAyd added the Groupby label Jun 5, 2018
@@ -684,8 +684,14 @@ def _gotitem(self, key, ndim, subset=None):
# with the same groupby
kwargs = dict([(attr, getattr(self, attr))
for attr in self._attributes])

if self._groupby.ndim == 1 and self._groupby.obj.name == key:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hate checks like this :>

does
groupby = self._groupby[key] not just work here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That’s what failed in the original code. Could alternately try and catch the error

@@ -520,6 +520,39 @@ def test_iter_raises(self, klass):
with pytest.raises(NotImplementedError):
iter(obj.rolling(2))

def test_agg(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can be a bit more descriptive here on the test name / expl

('low', 'mean'), ('low', 'max'), ('high', 'mean'),
('high', 'min')])
expected = pd.DataFrame([
[np.nan, np.nan, np.nan, np.nan],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maye put in TestApi. this should test rolling & expanding (atm we don't have a nice modular structure to do this :<)

@jreback jreback added the Reshaping Concat, Merge/Join, Stack/Unstack, Explode label Jun 5, 2018
@codecov
Copy link

codecov bot commented Jun 5, 2018

Codecov Report

Merging #21323 into master will increase coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #21323      +/-   ##
==========================================
+ Coverage   92.18%   92.19%   +<.01%     
==========================================
  Files         169      169              
  Lines       50819    50823       +4     
==========================================
+ Hits        46850    46855       +5     
+ Misses       3969     3968       -1
Flag Coverage Δ
#multiple 90.6% <100%> (ø) ⬆️
#single 42.37% <0%> (-0.01%) ⬇️
Impacted Files Coverage Δ
pandas/core/groupby/base.py 91.83% <100%> (+0.72%) ⬆️
pandas/core/base.py 97.61% <100%> (ø) ⬆️
pandas/core/strings.py 98.63% <0%> (ø) ⬆️
pandas/core/window.py 96.4% <0%> (+0.11%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6b1f460...c234b36. Read the comment docs.


try:
groupby = self._groupby[key] # get slice of frame
except Exception:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally would catch something more descriptive than an Exception, but that is all that gets raised:

raise Exception('Column(s) {selection} already selected'

Maybe an AttributeError would be more appropriate?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah see if u can catch something more specific bere

@WillAyd
Copy link
Member Author

WillAyd commented Jun 5, 2018

Test errors have to do with the column sorting of the returned frame. Do we make any guarantees about how that is supposed to be ordered across versions?

@jreback
Copy link
Contributor

jreback commented Jun 6, 2018

i would use an ordered dict for tests (so no guarantees on actual ordering)

Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very minor comments. pls rebase & merge on green.

pytest.raises(Exception, grouped['C'].__getitem__, 'D')

def test_as_index_series_column_slice_raises(df):
grouped = df.groupby('A', as_index=False)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add the issue number here

@@ -684,8 +684,14 @@ def _gotitem(self, key, ndim, subset=None):
# with the same groupby
kwargs = dict([(attr, getattr(self, attr))
for attr in self._attributes])

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor, but can you put the comments before the try (and consolidate them into 1)

@jreback jreback added this to the 0.24.0 milestone Jun 19, 2018
@jreback
Copy link
Contributor

jreback commented Sep 25, 2018

@WillAyd can you rebase

@pep8speaks
Copy link

pep8speaks commented Sep 25, 2018

Hello @WillAyd! Thanks for updating the PR.

Comment last updated on September 25, 2018 at 19:09 Hours UTC

@WillAyd
Copy link
Member Author

WillAyd commented Sep 26, 2018

@jreback all green

@jreback jreback merged commit 739e6be into pandas-dev:master Sep 26, 2018
@jreback
Copy link
Contributor

jreback commented Sep 26, 2018

thanks @WillAyd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Groupby Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG/ENH: groupby.rolling.agg / Column(s) low already selected
3 participants