feat: Implement reader that returns the last row of each series #4354
Conversation
WalkthroughThis update introduces a module for testing row selection within a time series dataset, a Changes
Sequence DiagramssequenceDiagram
participant Test as row_selector_test.rs
participant Mito as engine.rs
participant LastRow as last_row.rs
participant Scan as scan_region.rs
participant SeqScan as seq_scan.rs
Test->>Mito: Add row_selector_test module for testing
Test->>Mito: Implement test functions for last row selection
Mito->>LastRow: Introduce LastRowReader struct
Note over LastRow: Provides new method and next_last_row method
Scan->>SeqScan: Update scan condition to check series_row_selector
SeqScan->>LastRow: Add LastRowReader and modify reader selection logic
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (6)
- src/mito2/src/engine.rs (1 hunks)
- src/mito2/src/engine/row_selector_test.rs (1 hunks)
- src/mito2/src/read.rs (1 hunks)
- src/mito2/src/read/last_row.rs (1 hunks)
- src/mito2/src/read/scan_region.rs (1 hunks)
- src/mito2/src/read/seq_scan.rs (3 hunks)
Files skipped from review due to trivial changes (1)
- src/mito2/src/engine.rs
Additional comments not posted (11)
src/mito2/src/engine/row_selector_test.rs (3)
96-99: LGTM!The function
test_last_row_append_mode_disabledcorrectly tests theLastRowReaderwith append mode disabled by callingtest_last_rowwithappend_modeset tofalse.
101-104: LGTM!The function
test_last_row_append_mode_enabledcorrectly tests theLastRowReaderwith append mode enabled by callingtest_last_rowwithappend_modeset totrue.
27-94: Verify the function usage and ensure comprehensive test coverage.The function
test_last_rowsets up the environment, creates the engine, and tests the last row reader by inserting rows and flushing regions. Ensure that this function is used correctly in all relevant test cases and consider adding edge cases such as empty datasets or datasets with only one series.Verification successful
Verify comprehensive test coverage for
test_last_rowfunctionThe function
test_last_rowis invoked in two specific test cases:test_last_row_append_mode_disabledandtest_last_row_append_mode_enabled. Ensure these test cases cover a wide range of scenarios, including edge cases such as empty datasets or datasets with only one series.
src/mito2/src/engine/row_selector_test.rs: test_last_row_append_mode_disabledsrc/mito2/src/engine/row_selector_test.rs: test_last_row_append_mode_enabledScripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the usage of `test_last_row` function and ensure comprehensive test coverage. # Test: Search for the function usage. Expect: Only occurances of the function usage in relevant test cases. rg --type rust -A 5 $'test_last_row'Length of output: 1753
src/mito2/src/read/last_row.rs (5)
36-43: LGTM!The method
LastRowReader::newcorrectly initializes theLastRowReaderwith the provided reader and setslast_batchtoNone.
45-72: LGTM!The method
LastRowReader::next_last_rowcorrectly iterates over batches and returns the last row of the next key, handling different keys and empty batches appropriately.
75-80: LGTM!The implementation of
BatchReaderforLastRowReadercorrectly delegates thenext_batchmethod tonext_last_row.
89-115: LGTM!The test
test_last_row_one_batchcorrectly checks the functionality ofLastRowReaderwith a single batch, ensuring it returns the last row.
117-152: LGTM!The test
test_last_row_multi_batchcorrectly checks the functionality ofLastRowReaderwith multiple batches, ensuring it returns the last row for each key.src/mito2/src/read/seq_scan.rs (1)
Line range hint
214-234:
LGTM!The segment correctly integrates the
LastRowReaderand updates the reader selection logic to handle deduplication and series row selector settings.src/mito2/src/read/scan_region.rs (1)
212-213: LGTM!The segment correctly updates the
scannermethod to handle the absence of a series row selector when the table is in append-only mode, ensuring that unordered scans are used appropriately.src/mito2/src/read.rs (1)
19-19: Addition oflast_rowmodule approved.The
last_rowmodule is appropriately marked withpub(crate)visibility for internal use.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #4354 +/- ##
==========================================
- Coverage 85.18% 84.91% -0.27%
==========================================
Files 1061 1063 +2
Lines 189447 189615 +168
==========================================
- Hits 161372 161007 -365
- Misses 28075 28608 +533 |
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
What's changed and what's your intention?
This PR implements
LastRowReaderthat only returns the last row of each series (Batcheswith the same primary key). This can reduce the number of rows to process in the query engine.When the scan requests has a
TimeSeriesRowSelectorhint, the storage engine will useSeqScanto scan data and enables theLastRowReader.Checklist
Summary by CodeRabbit
New Features
Bug Fixes
Tests