Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable parquet page level skipping (page index pruning) by default #4085

Closed
4 tasks done
Tracked by #3462
alamb opened this issue Nov 2, 2022 · 1 comment · Fixed by #5099
Closed
4 tasks done
Tracked by #3462

Enable parquet page level skipping (page index pruning) by default #4085

alamb opened this issue Nov 2, 2022 · 1 comment · Fixed by #5099
Assignees

Comments

@alamb
Copy link
Contributor

alamb commented Nov 2, 2022

In #3834 and #847 @Ted-Jiang added support for skipping rows in the parquet scan via the RowIndex mechanism 🎉

This feature is currently enabled via a feature flag, which is disabled by default.

This ticket tracks enabling this feature by default.

Currently known items are:

Related to #3463

@alamb
Copy link
Contributor Author

alamb commented Mar 27, 2023

BTW update here is the latest benchmark runs I did -- #5099 (comment) -- show some small performance regressions we need to fix in order to turn this on by default

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment