Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor GroupBy init to avoid factorization #6689

Closed
wants to merge 163 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
163 commits
Select commit Hold shift + click to select a range
c486df7
Move `_reduce_method` classmethod to `groupby.py` module
andersy005 Aug 12, 2021
ef91e6e
Add _numpy_groupies module
andersy005 Aug 13, 2021
511dd44
Add more aggregations
andersy005 Aug 13, 2021
3ee6200
Remove comments
andersy005 Aug 13, 2021
f088392
Fix position keyword arguments
andersy005 Aug 13, 2021
35944e4
Remove `_numpy_groupies.py` module
andersy005 Aug 13, 2021
9be0228
Merge branch 'pydata:main' into groupby-aggs-using-numpy-groupies
andersy005 Aug 24, 2021
e6bcce9
some fixes
dcherian Aug 24, 2021
4702c9d
Merge branch 'groupby-aggs-using-numpy-groupies' of github.com:anders…
dcherian Aug 24, 2021
489b2ff
Merge branch 'main' into groupby-aggs-using-numpy-groupies
andersy005 Sep 26, 2021
cdf7612
Fix resample test
dcherian Oct 3, 2021
af4cc5d
Fix reduce methods
dcherian Oct 3, 2021
58c1c6b
Add _dask_groupby_kwargs
dcherian Oct 3, 2021
69fd563
Avoid forwarding DummyGroup objects
dcherian Oct 3, 2021
b1e3ab2
Raise error when reducing along indexed dimensions with squeeze=True
dcherian Oct 3, 2021
462e61b
Don't pass numeric_only to DataArray.reduce
dcherian Oct 4, 2021
1d9a360
Add CI for now
dcherian Oct 4, 2021
f4748ee
typo
dcherian Oct 4, 2021
b97ffcb
Fix windows env
dcherian Oct 4, 2021
9b44db9
Fix keep_attrs test
dcherian Oct 4, 2021
262a3f5
Update ci/requirements/environment-windows.yml
dcherian Oct 4, 2021
e3b3a00
Fix resampling
dcherian Oct 5, 2021
4b25db5
Merge branch 'groupby-aggs-using-numpy-groupies' of github.com:anders…
dcherian Oct 5, 2021
0f2c59f
Merge branch 'main' of github.com:pydata/xarray into groupby-aggs-usi…
andersy005 Oct 26, 2021
77f0e0e
Merge remote-tracking branch 'upstream/main' into groupby-aggs-using-…
dcherian Nov 5, 2021
faee02c
fix env stuff + remove env var
dcherian Nov 5, 2021
3608e9f
get working again
dcherian Nov 5, 2021
ad25f78
Add to asv env
dcherian Nov 6, 2021
932b9a5
Separate out median
dcherian Nov 6, 2021
ac85e72
make dask_groupby actually optional
dcherian Nov 6, 2021
d238459
any,all
dcherian Nov 6, 2021
a2168df
typo again
dcherian Nov 6, 2021
6b9a81a
Better generator for reductions.
dcherian Nov 8, 2021
569c67f
Add ddof for var, std
dcherian Nov 8, 2021
816e794
Generate DataArray, Dataset reductions too.
dcherian Nov 8, 2021
a04ed82
Small changes
dcherian Nov 8, 2021
7f39cc0
Minor docstring improvements.
dcherian Nov 8, 2021
99bfe12
Fixes #5898
dcherian Nov 8, 2021
08911b9
Merge remote-tracking branch 'upstream/main' into groupby-aggs-using-…
dcherian Nov 8, 2021
9bb2c32
Reorder docstring to match numpy
dcherian Nov 8, 2021
dea8fd9
REfactor
dcherian Nov 8, 2021
f06e6a7
Revert "Separate out median"
dcherian Nov 8, 2021
0661c1b
Refactored generator
dcherian Nov 8, 2021
0c35c0c
Reimplemented
dcherian Nov 8, 2021
3e08964
Add benchmarks
dcherian Nov 8, 2021
583187a
Fix benchmark to not groupby chunked variables.
dcherian Nov 9, 2021
4ef53db
Start supporting ndim groups
dcherian Nov 9, 2021
6afb3bf
WIP refactor init
dcherian Nov 9, 2021
35af40a
Revert "WIP refactor init"
dcherian Nov 9, 2021
c9a82b3
Revert "Start supporting ndim groups"
dcherian Nov 9, 2021
0ac5498
Avoid stacking by default
dcherian Nov 9, 2021
b9bc1dd
Update reductions
dcherian Nov 10, 2021
bece14e
Fix median and add test.
dcherian Nov 10, 2021
41f0aa5
fix test
dcherian Nov 10, 2021
0559ee1
Fix var, std doctests
dcherian Nov 10, 2021
31e1fd2
Force test failure to check CI env
dcherian Nov 10, 2021
47b593c
Use conda-forge numpy_groupies in CI
dcherian Nov 10, 2021
c7e9d96
Minor improvement
dcherian Nov 10, 2021
77d2665
Revert "Force test failure to check CI env"
dcherian Nov 10, 2021
11c3d33
Fixed doctests in dask_groupby
dcherian Nov 10, 2021
be53f13
See if its an import error
dcherian Nov 10, 2021
35908b5
Revert "See if its an import error"
dcherian Nov 10, 2021
9c2cbb8
Ppass through objects with only numpy or dask arrays
dcherian Nov 11, 2021
e9af57c
Try fixing mypy
dcherian Nov 11, 2021
415eb29
Fix bug when binning by nD variable.
dcherian Nov 12, 2021
edbd376
Fix binning and weird issues with precision and pd.cut
dcherian Nov 14, 2021
c189eea
Fix upsampling with resample
dcherian Nov 14, 2021
43ade8c
"blockwise" need not be the best strategy for resample..
dcherian Nov 15, 2021
553735e
Merge remote-tracking branch 'upstream/main' into groupby-aggs-using-…
dcherian Nov 15, 2021
03b7b31
one more bugfix
dcherian Nov 15, 2021
e038cc7
Fix dimension order when binning a dimension coordinate
dcherian Nov 15, 2021
1f370f6
silence warning
dcherian Nov 15, 2021
7375dd4
fix test.
dcherian Nov 15, 2021
860f7be
add extra test
dcherian Nov 15, 2021
ced9034
Update upstream-dev env
dcherian Nov 15, 2021
b269439
[test-upstream] Revert setting npg option in benchmarks
dcherian Nov 16, 2021
cc8abfe
[test-upstream] Rename to flox
dcherian Nov 16, 2021
033f5b5
Add to print_versions
dcherian Nov 18, 2021
bd24db4
Add to all-but-dask
dcherian Nov 18, 2021
098467d
Force failure to make sure CI is working.
dcherian Nov 18, 2021
a282ad4
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 18, 2021
8f23310
Revert "Force failure to make sure CI is working."
dcherian Nov 19, 2021
5dcb5bf
Attempt fixing typing errors
Illviljan Nov 20, 2021
411d75d
Now get normal code running as well
Illviljan Nov 20, 2021
6a9a124
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 20, 2021
dd28a57
updates
dcherian Nov 20, 2021
f03b675
Merge branch 'main' into pr/5950
Illviljan Nov 20, 2021
dfbe103
Merge branch 'generate-reductions-class' of https://github.com/dcheri…
Illviljan Nov 20, 2021
2bbddaf
make reduce args consistent
Illviljan Nov 20, 2021
3d854e5
more reduce edits
Illviljan Nov 20, 2021
be33560
one more reduce
Illviljan Nov 20, 2021
0f94bec
another reduce
Illviljan Nov 20, 2021
19d82cd
more reduce
Illviljan Nov 20, 2021
cd8a898
add doctests
dcherian Nov 20, 2021
4f378a3
Bugfix DataArray resampling.
dcherian Nov 22, 2021
6916fa7
Update xarray/util/generate_reductions.py
dcherian Nov 22, 2021
af03ca4
Small improvement to resampling
dcherian Nov 26, 2021
cfd2c07
minimize conflicts
dcherian Nov 26, 2021
3c51b1a
Squash merge #5950
dcherian Nov 26, 2021
2a1b12f
Update xarray/util/generate_reductions.py
dcherian Nov 26, 2021
638d98a
Merge remote-tracking branch 'upstream/main' into groupby-aggs-using-…
dcherian Nov 26, 2021
45feeab
Annotate some reduction tests.
dcherian Nov 26, 2021
b406789
Merge remote-tracking branch 'upstream/main' into generate-reductions…
dcherian Nov 26, 2021
1875fd2
Merge branch 'main' into groupby-aggs-using-numpy-groupies
andersy005 Dec 16, 2021
66151f6
Merge branch 'main' into pr/5950
Illviljan Dec 21, 2021
3dc94ae
force keyword args after dim
Illviljan Dec 21, 2021
bc55db3
Write to file using open() instead.
Illviljan Dec 21, 2021
b78df18
Update _reductions.py
Illviljan Dec 21, 2021
16372a5
Merge branch 'generate-reductions-class' of https://github.com/dcheri…
Illviljan Dec 21, 2021
74064b9
manual tweaks to make ci happy
Illviljan Dec 21, 2021
8336c53
Merge branch 'main' into pr/5950
Illviljan Dec 27, 2021
ad6b5bc
Merge branch 'main' into groupby-aggs-using-numpy-groupies
dcherian Dec 28, 2021
e1ba8a2
use_numpy_groupies → use_flox
dcherian Dec 28, 2021
bdb999f
fix tests
dcherian Dec 29, 2021
4fb17b1
Merge remote-tracking branch 'upstream/main' into groupby-aggs-using-…
dcherian Dec 29, 2021
70266e1
fix tests
dcherian Dec 29, 2021
41e43fe
fix tests
dcherian Dec 30, 2021
2c2e7dc
Merge remote-tracking branch 'upstream/main' into groupby-aggs-using-…
dcherian Jan 12, 2022
c157fca
Merge remote-tracking branch 'upstream/main' into groupby-aggs-using-…
dcherian Jan 13, 2022
3f3a197
Merge branch 'main' into groupby-aggs-using-numpy-groupies
andersy005 Jan 13, 2022
cd51a15
Merge branch 'main' into generate-reductions-class
dcherian Feb 16, 2022
7b34077
Fix path
dcherian Feb 16, 2022
9799d87
Apply suggestions from code review
dcherian Mar 8, 2022
1fcd080
Fixes
dcherian Mar 8, 2022
ebe9985
Merge branch 'main' into generate-reductions-class
dcherian Mar 9, 2022
d5f627c
update _reductions
dcherian Mar 9, 2022
e348c76
Merge branch 'main' into groupby-aggs-using-numpy-groupies
dcherian Mar 9, 2022
434db03
polish
dcherian Mar 10, 2022
62474a8
Merge branch 'generate-reductions-class' into groupby-aggs-using-nump…
dcherian Mar 10, 2022
94bcb32
Merge branch 'main' into groupby-aggs-using-numpy-groupies
dcherian Mar 13, 2022
a1769ba
polish
dcherian Mar 13, 2022
26d85d5
loooser test
dcherian Mar 13, 2022
705b3f0
Merge remote-tracking branch 'upstream/main' into groupby-aggs-using-…
dcherian Mar 13, 2022
e412583
Fix.
dcherian Mar 13, 2022
2694dbe
Test flox kwargs
dcherian Mar 13, 2022
1a91802
Merge branch 'main' into groupby-aggs-using-numpy-groupies
dcherian Mar 29, 2022
a2518b3
fix
dcherian Mar 30, 2022
628e33d
Test cleanup
dcherian Mar 30, 2022
5edd319
WIP
dcherian Mar 30, 2022
aa64c43
groupby binary ops: remove need for stacking
dcherian Mar 30, 2022
299e6c9
WIP
dcherian Mar 30, 2022
1277544
Merge branch 'main' into groupby-refactor-init
dcherian Jun 5, 2022
dcc5328
fix merge
dcherian Jun 5, 2022
c466c57
Fix quantile tests.
dcherian Jun 5, 2022
e3fd36f
Fix empty group error
dcherian Jun 5, 2022
bfb0b27
Fix datetime binning
dcherian Jun 5, 2022
7d83c02
Disable some error message
dcherian Jun 5, 2022
553ef48
WIP
dcherian Jun 5, 2022
8aa3e2f
temp fix for binning
dcherian Jun 5, 2022
9c5a691
Better fix for quantile tests
dcherian Jun 5, 2022
14c7853
Fix some resample tests.
dcherian Jun 5, 2022
19de080
Fix more resample tests
dcherian Jun 5, 2022
cb17efa
More fixes
dcherian Jun 5, 2022
565b37c
Fix binning
dcherian Jun 5, 2022
73d4f4d
Fix binary ops
dcherian Jun 5, 2022
95189e3
Fix groupby_bins math
dcherian Jun 5, 2022
23f0996
Fix final bins test!
dcherian Jun 5, 2022
bb2da3c
Fix bad merge
dcherian Jun 5, 2022
dbaf3e1
Warn when grouping by dask array
dcherian Jun 12, 2022
492b9d6
Refactor out init resampling-specific logic
dcherian Jun 12, 2022
35152d7
Merge branch 'main' into groupby-refactor-init
dcherian Jun 12, 2022
cac2249
Bring back error check for no data in bins
dcherian Jun 12, 2022
6bd276c
Merge branch 'main' into groupby-refactor-init
dcherian Jun 12, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions xarray/core/computation.py
Original file line number Diff line number Diff line change
Expand Up @@ -525,6 +525,8 @@ def apply_groupby_func(func, *args):
from .variable import Variable

groupbys = [arg for arg in args if isinstance(arg, GroupBy)]
for g in groupbys:
g._initialize_old()
assert groupbys, "must have at least one groupby to iterate over"
first_groupby = groupbys[0]
if any(not first_groupby._group.equals(gb._group) for gb in groupbys[1:]):
Expand Down
Loading