Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PERF-#5087: use cache for widths/lengths/index/columns if possible #5031

Merged
merged 3 commits into from
Oct 10, 2022
Merged

PERF-#5087: use cache for widths/lengths/index/columns if possible #5031

merged 3 commits into from
Oct 10, 2022

Conversation

anmyachev
Copy link
Collaborator

@anmyachev anmyachev commented Sep 24, 2022 •
edited
Loading

Signed-off-by: Myachev anatoly.myachev@intel.com

What do these changes do?

  • commit message follows format outlined here
  • passes flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py
  • passes black --check modin/ asv_bench/benchmarks scripts/doc_checker.py
  • signed commit with git commit -s
  • Resolves PERF: use cache for widths/lengths/index/columns if possible #5087
  • tests added and passing
  • module layout described at docs/development/architecture.rst is up-to-date
  • added (Issue Number: PR title (PR Number)) and github username to release notes for next major release

@codecov
Copy link

codecov bot commented Sep 24, 2022 •
edited
Loading

Codecov Report

Merging #5031 (2a145ea) into master (0215a13) will increase coverage by 5.03%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #5031      +/-   ##
==========================================
+ Coverage   84.56%   89.60%   +5.03%     
==========================================
  Files         256      257       +1     
  Lines       19346    19629     +283     
==========================================
+ Hits        16360    17588    +1228     
+ Misses       2986     2041     -945
Impacted Files Coverage Δ
modin/core/dataframe/pandas/dataframe/dataframe.py 95.23% <ø> (ø)
modin/logging/config.py 94.59% <0.00%> (-1.30%) ⬇️
modin/experimental/batch/test/test_pipeline.py 90.21% <0.00%> (ø)
modin/pandas/series.py 94.51% <0.00%> (+0.23%) ⬆️
modin/pandas/base.py 95.26% <0.00%> (+0.26%) ⬆️
...odin/core/storage_formats/pandas/query_compiler.py 96.33% <0.00%> (+0.39%) ⬆️
modin/pandas/series_utils.py 98.89% <0.00%> (+0.55%) ⬆️
...mentations/pandas_on_ray/partitioning/partition.py 91.59% <0.00%> (+0.84%) ⬆️
modin/core/io/text/excel_dispatcher.py 94.01% <0.00%> (+0.85%) ⬆️
modin/pandas/groupby.py 93.83% <0.00%> (+0.94%) ⬆️
... and 38 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@anmyachev anmyachev changed the title PERF-#0000: use cache for widths/lengths if possible PERF-#5087: use cache for widths/lengths if possible Oct 5, 2022
@anmyachev anmyachev marked this pull request as ready for review October 6, 2022 16:22
@anmyachev anmyachev requested a review from a team as a code owner October 6, 2022 16:22
vnlitvinov
vnlitvinov previously approved these changes Oct 6, 2022
View reviewed changes
Copy link
Collaborator

@vnlitvinov vnlitvinov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Got any benchmarks to support that?

@anmyachev
Copy link
Collaborator Author

LGTM!

Got any benchmarks to support that?

To do this, we need to test several operations in a row. There are no such benchmarks yet :(

YarShev
YarShev previously approved these changes Oct 7, 2022
View reviewed changes
@YarShev YarShev dismissed stale reviews from vnlitvinov and themself via 1d88dee October 7, 2022 08:21
vnlitvinov
vnlitvinov previously approved these changes Oct 7, 2022
View reviewed changes
Copy link
Collaborator

@vnlitvinov vnlitvinov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

anmyachev and others added 3 commits October 7, 2022 12:25
vnlitvinov
vnlitvinov reviewed Oct 7, 2022
View reviewed changes
Copy link
Collaborator

@vnlitvinov vnlitvinov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix PR title; code changes look good

@anmyachev anmyachev changed the title PERF-#5087: use cache for widths/lengths if possible PERF-#5087: use cache for widths/lengths/index/columns if possible Oct 7, 2022
@YarShev
Copy link
Collaborator

YarShev commented Oct 7, 2022

ci / test-asv-benchmarks (pull_request) failed. Take a look please.

@anmyachev
Copy link
Collaborator Author

ci / test-asv-benchmarks (pull_request) failed. Take a look please.

restart tests fixes it

@YarShev YarShev merged commit 00a0fb9 into modin-project:master Oct 10, 2022
@anmyachev anmyachev deleted the use-cache branch October 10, 2022 08:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vnlitvinov vnlitvinov vnlitvinov left review comments

@YarShev YarShev YarShev approved these changes

Assignees
No one assigned
Labels
Ready for review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

PERF: use cache for widths/lengths/index/columns if possible
3 participants
@anmyachev @YarShev @vnlitvinov