Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider to avoid force_materialization call in to_pandas function #5269

Open
anmyachev opened this issue Nov 25, 2022 · 0 comments
Open

Consider to avoid force_materialization call in to_pandas function #5269

anmyachev opened this issue Nov 25, 2022 · 0 comments
Labels
Internals Internal modin functionality P3 Very minor bugs, or features we can hopefully add some day. Performance 🚀 Performance related issues and pull requests.

Comments

@anmyachev
Copy link
Collaborator

force_materialization function takes dataframes from storage, merges them into one and puts them back into storage. The step at which the object is put back into storage can be avoided, resulting in a performance boost. However, this can significantly complicate to_pandas function , so when implementing it someone needs to understand whether the resulting performance gain is worth the increased code complexity. Originally found in #4776 (comment)

@anmyachev anmyachev added Performance 🚀 Performance related issues and pull requests. Internals Internal modin functionality P3 Very minor bugs, or features we can hopefully add some day. labels Nov 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Internals Internal modin functionality P3 Very minor bugs, or features we can hopefully add some day. Performance 🚀 Performance related issues and pull requests.
Projects
None yet
Development

No branches or pull requests

1 participant