Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Avoid deep-copy when constructing groupby codes #9429

Merged
merged 1 commit into from
Sep 5, 2024
Merged

Avoid deep-copy when constructing groupby codes #9429

merged 1 commit into from
Sep 5, 2024

Conversation

dcherian
Copy link
Contributor

@dcherian dcherian commented Sep 4, 2024 •
edited
Loading 

| Change   | Before [a8c98967]    | After [948366dc]    |   Ratio | Benchmark (Parameter)                            |
|----------|----------------------|---------------------|---------|--------------------------------------------------|
| -        | 117±0.6ms            | 14.8±0.2ms          |    0.13 | groupby.GroupByLongTime.time_mean(True, False)   |
| -        | 115±0.6ms            | 14.0±0.1ms          |    0.12 | groupby.GroupByLongTime.time_mean(False, False)  |
| -        | 113±0.9ms            | 9.23±0.3ms          |    0.08 | groupby.GroupByLongTime.time_mean(True, True)    |
| -        | 112±0.6ms            | 8.28±0.2ms          |    0.07 | groupby.GroupByLongTime.time_mean(False, True)   |
| -        | 105±0.5ms            | 3.45±0.03ms         |    0.03 | groupby.GroupByLongTime.time_setup(True, False)  |
| -        | 105±0.8ms            | 3.49±0.02ms         |    0.03 | groupby.GroupByLongTime.time_setup(True, True)   |
| -        | 105±0.9ms            | 2.40±0.03ms         |    0.02 | groupby.GroupByLongTime.time_setup(False, False) |
| -        | 105±1ms              | 2.45±0.01ms         |    0.02 | groupby.GroupByLongTime.time_setup(False, True)  |

@dcherian dcherian added the plan to merge Final call for comments label Sep 4, 2024
@dcherian dcherian merged commit 4013ffa into pydata:main Sep 5, 2024
41 of 43 checks passed
dcherian added a commit to dcherian/xarray that referenced this pull request Sep 17, 2024
@dcherian
Merge branch 'main' into groupby-shuffle
0679d2b 
* main: (29 commits)
  Release notes for v2024.09.0 (pydata#9480)
  Fix `DataTree.coords.__setitem__` by adding `DataTreeCoordinates` class (pydata#9451)
  Rename DataTree's "ds" and "data" to "dataset" (pydata#9476)
  Update DataTree repr to indicate inheritance (pydata#9470)
  Bump pypa/gh-action-pypi-publish in the actions group (pydata#9460)
  Repo checker (pydata#9450)
  Add days_in_year and decimal_year to dt accessor (pydata#9105)
  remove parent argument from DataTree.__init__ (pydata#9465)
  Fix inheritance in DataTree.copy() (pydata#9457)
  Implement `DataTree.__delitem__` (pydata#9453)
  Add ASV for datatree.from_dict (pydata#9459)
  Make the first argument in DataTree.from_dict positional only (pydata#9446)
  Fix typos across the code, doc and comments (pydata#9443)
  DataTree should not be "Generic" (pydata#9445)
  Disallow passing a DataArray as data into the DataTree constructor (pydata#9444)
  Support additional dtypes in `resample` (pydata#9413)
  Shallow copy parent and children in DataTree constructor (pydata#9297)
  Bump minimum versions for dependencies (pydata#9434)
  Always include at least one category in random test data (pydata#9436)
  Avoid deep-copy when constructing groupby codes (pydata#9429)
  ...
@dcherian dcherian deleted the speedup-factorize branch September 19, 2024 05:25
hollymandel pushed a commit to hollymandel/xarray that referenced this pull request Sep 23, 2024
@dcherian @hollymandel
Avoid deep-copy when constructing groupby codes (pydata#9429)
2d14d03 
Closes pydata#9426
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@max-sixty max-sixty max-sixty approved these changes

Assignees
No one assigned
Labels
plan to merge Final call for comments run-benchmark Run the ASV benchmark workflow topic-groupby topic-performance
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Grouping is significantly slower when adding auxiliary coordinates to the time dimension
2 participants
@dcherian @max-sixty