-
Notifications
You must be signed in to change notification settings - Fork 308
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: support pattern
as pipeline key name
#4368
chore: support pattern
as pipeline key name
#4368
Conversation
WalkthroughThe updates make significant refinements to the ETL pipeline in the project. Primarily, the changes involve enhancing pattern processing capabilities for dissecting and regex functionalities, introducing new methods and constants, and improving test coverage to ensure the correctness of these features. Additionally, test files were refactored to use a common function for column schema creation, promoting code reusability and simplifying the codebase. Changes
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
pattern
as pipeline key name with patterns
pattern
as pipeline key name
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (8)
- src/pipeline/src/etl/processor/dissect.rs (2 hunks)
- src/pipeline/src/etl/processor/regex.rs (3 hunks)
- src/pipeline/tests/common.rs (2 hunks)
- src/pipeline/tests/dissect.rs (1 hunks)
- src/pipeline/tests/gsub.rs (2 hunks)
- src/pipeline/tests/join.rs (1 hunks)
- src/pipeline/tests/on_failure.rs (5 hunks)
- src/pipeline/tests/regex.rs (1 hunks)
Additional comments not posted (13)
src/pipeline/tests/gsub.rs (1)
16-16
: LGTM!The changes improve code reuse by using the
make_column_schema
function from thecommon
module.src/pipeline/tests/common.rs (1)
15-15
: LGTM!The addition of the
SemanticType
parameter to themake_column_schema
function enhances the accuracy of the test schema.src/pipeline/tests/join.rs (1)
35-44
: LGTM!The changes improve code reuse by using the
make_column_schema
function from thecommon
module.src/pipeline/tests/regex.rs (1)
1-109
: LGTM!The changes improve code reuse by using the
make_column_schema
function from thecommon
module.src/pipeline/tests/dissect.rs (2)
20-65
: LGTM!The test function
test_dissect_pattern
is well-structured and validates the functionality as expected.
67-113
: LGTM!The test function
test_dissect_patterns
is well-structured and validates the functionality as expected.src/pipeline/tests/on_failure.rs (4)
Line range hint
16-48
: LGTM!The test function
test_on_failure_with_ignore
is well-structured and validates the functionality as expected.
Line range hint
49-93
: LGTM!The test function
test_on_failure_with_default
is well-structured and validates the functionality as expected.
Line range hint
94-129
: LGTM!The test function
test_default
is well-structured and validates the functionality as expected.
Line range hint
130-181
: LGTM!The test function
test_multiple_on_failure
is well-structured and validates the functionality as expected.src/pipeline/src/etl/processor/regex.rs (2)
160-162
: LGTM!The method
try_with_patterns
is well-structured and handles errors appropriately.
216-243
: LGTM!The test function
test_simple_parse
is well-structured and validates the functionality as expected.src/pipeline/src/etl/processor/dissect.rs (1)
562-565
: LGTM!The method
try_from
forDissectProcessor
is well-structured and handles errors appropriately.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #4368 +/- ##
==========================================
- Coverage 85.21% 84.87% -0.34%
==========================================
Files 1060 1063 +3
Lines 189484 190148 +664
==========================================
- Hits 161474 161395 -79
- Misses 28010 28753 +743 |
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
What's changed and what's your intention?
This pr mainly
pattern
as field key name in pipeline yaml along withpatterns
Checklist
Summary by CodeRabbit
New Features
Tests