-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FEAT-#5394: Reduce amount of remote calls for TreeReduce and GroupByReduce operators #11
Conversation
…educe operators Signed-off-by: Kirill Suvorov <kirill.suvorov@intel.com>
Clone of the PR modin-project/modin#7245 |
My review is in progress 📖 - I will have feedback for you in a few minutes! |
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have reviewed your code and did not find any issues!
Please note that I can make mistakes, and you should still encourage your team to review your code as well.
Apply approaches from PR-7136 for TreeReduce and GroupByReduce operators
What do these changes do?
flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py
black --check modin/ asv_bench/benchmarks scripts/doc_checker.py
git commit -s
docs/development/architecture.rst
is up-to-dateDescription by Korbit AI
Note
This feature is in early access. You can enable or disable it in the Korbit Console.
FEAT-#5394: Reduce amount of remote calls for TreeReduce and GroupByReduce operators
Refactor the
map
function indataframe.py
andmap_partitions
function inpartition_manager.py
to simplify the logic and reduce the number of remote calls.These changes are being made to improve the performance of the
TreeReduce
andGroupByReduce
operators by reducing the complexity and number of remote calls. The previous implementation had conditional logic based on the number of CPU cores, which has been streamlined to always use block-wise mapping, thus simplifying the code and potentially reducing overhead.