-
Notifications
You must be signed in to change notification settings - Fork 308
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: skip caching uncompressed pages if they are large #4705
Conversation
This reverts commit 4618f57.
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4705 +/- ##
==========================================
- Coverage 84.81% 84.53% -0.28%
==========================================
Files 1113 1113
Lines 200467 200550 +83
==========================================
- Hits 170022 169540 -482
- Misses 30445 31010 +565 |
@coderabbitai summary |
Actions performedSummary regeneration triggered. |
WalkthroughThe changes involve significant modifications to the caching mechanism, metrics tracking, and logging within the codebase. Key updates include the renaming of structures and enums for clarity, the introduction of new methods for creating instances, and enhancements to metrics for better performance monitoring. Additionally, the output formatting of execution plans has been adjusted for consistency. These changes collectively aim to improve code clarity, usability, and observability across various components. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Cache
participant Metrics
participant Reader
User->>Cache: Request Page
Cache->>Reader: Fetch Page
Reader->>Metrics: Record Fetch Cost
Metrics-->>Cache: Provide Metrics
Cache-->>User: Return Page
Tip Announcements
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
…#4705) * feat: cache each uncompressed page * chore: remove unused function * chore: log * chore: log * chore: row group pages cache kv * feat: also support row group level cache * chore: fix range count * feat: don't cache compressed page for row group cache * feat: use function to get part * chore: log whether scan is from compaction * chore: avoid get column * feat: add timer metrics * chore: Revert "feat: add timer metrics" This reverts commit 4618f57. * feat: don't cache individual uncompressed page * feat: append in row group level under append mode Signed-off-by: Ruihang Xia <waynestxia@gmail.com> * chore: fetch pages cost * perf: yield * Update src/mito2/src/sst/parquet/row_group.rs * refactor: cache key * feat: print file num and row groups num in explain * test: update sqlness test * chore: Update src/mito2/src/sst/parquet/page_reader.rs --------- Signed-off-by: Ruihang Xia <waynestxia@gmail.com> Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
What's changed and what's your intention?
This PR skips caching uncompressed pages for a column in a row group if it has multiple uncompressed pages. Since caching uncompressed pages for this column may be memory inefficient and decompressing all pages may be slower than only decompress pages on demand.
It also prints more information to logs and explain outputs.
Checklist
Summary by CodeRabbit
New Features
ScanPartList
struct.Bug Fixes
Documentation
Style