Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add EXTRACT builtin function; update partiql-tests #340

Merged
merged 5 commits into from
Apr 17, 2023
Merged

Conversation

alancai98
Copy link
Member

@alancai98 alancai98 commented Apr 13, 2023

Implements EXTRACT builtin function and updates to the latest partiql-tests. Also fixes some parsing of the datetime part for YEAR, TIMEZONE_HOUR, and TIMEZONE_MINUTE. Makes the following assumptions:

  • EXTRACT with a date component (year, month, day) on a time -> returns missing
  • EXTRACT with a time component (hour, minute, second) -> returns missing
  • a timezone part (timezone_hour, timezone_minute) with a datetime value without a timezone?

All of the above assumptions aren't specified in any SQL spec to my knowledge. The second assumption differs from what partiql-lang-kotlin currently does (which defines a default of 0). Created a spec issue to track: partiql/partiql-lang#14 along with a conformance test issue: partiql/partiql-tests#83.

Another point that may differ from partiql-lang-kotlin is the data type returned by the EXTRACT function. partiql-lang-kotlin always returns a decimal, while this PR returns a decimal for seconds and integer otherwise. The SQL spec says that this is implementation-defined. Add a followup comment in the above spec issue: partiql/partiql-lang#14


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@alancai98 alancai98 self-assigned this Apr 13, 2023
@codecov
Copy link

codecov bot commented Apr 13, 2023

Codecov Report

Patch coverage: 76.61% and project coverage change: -0.10 ⚠️

Comparison is base (2cb413a) 81.30% compared to head (73a47be) 81.20%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #340      +/-   ##
==========================================
- Coverage   81.30%   81.20%   -0.10%     
==========================================
  Files          44       44              
  Lines       10261    10521     +260     
==========================================
+ Hits         8343     8544     +201     
- Misses       1918     1977      +59     
Impacted Files Coverage Δ
partiql-logical/src/lib.rs 46.21% <ø> (ø)
partiql-parser/src/preprocessor.rs 92.92% <ø> (ø)
partiql-value/src/ion.rs 48.36% <57.14%> (+0.36%) ⬆️
partiql-eval/src/eval/expr/mod.rs 81.29% <61.90%> (-4.95%) ⬇️
partiql-value/src/lib.rs 95.04% <66.66%> (-0.22%) ⬇️
partiql-value/src/datetime.rs 61.90% <88.23%> (+2.64%) ⬆️
partiql-logical-planner/src/call_defs.rs 84.41% <89.65%> (+1.38%) ⬆️
partiql-eval/src/plan.rs 95.87% <100.00%> (+0.32%) ⬆️

... and 1 file with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@github-actions
Copy link

github-actions bot commented Apr 13, 2023

Conformance comparison report

Base (2cb413a) cbe8cd7 +/-
% Passing 81.59% 82.88% 1.29%
✅ Passing 2699 2760 61
❌ Failing 415 376 -39
🔶 Ignored 194 194 0
Total Tests 3308 3330 22

Number passing in both: 2650

Number failing in both: 314

Number passing in Base (2cb413a) but now fail: 0

Number failing in Base (2cb413a) but now pass: 10

The following test(s) were previously failing but now pass. Before merging, confirm they are intended to pass:

Click here to see
  • partiql_tests::eval::query::group_by::group_by::group_by::group_by_test::select_region_id_p_supplier_id_mixed_from_products_sparse_p_group_by_p_region_id_p_supplier_id_mixed
  • partiql_tests::eval::query::group_by::group_by::group_by::group_by_test::select_p_supplier_id_mixed_from_products_sparse_p_group_by_p_supplier_id_mixed
  • partiql_tests::eval::query::group_by::group_by::group_by::group_by_test::select_supplier_id_missings_from_products_sparse_p_group_by_p_supplier_id_missings
  • partiql_tests::eval::query::group_by::group_by::group_by::group_by_test::select_region_id_supplier_id_missings_from_products_sparse_p_group_by_p_region_id_p_supplier_id_missings
  • partiql_tests::eval::query::group_by::group_by::group_by::group_by_test::select_p_supplier_id_missings_from_products_sparse_p_group_by_p_supplier_id_missings
  • partiql_tests::eval::query::group_by::group_by::group_by::group_by_test::select_region_id_supplier_id_mixed_from_products_sparse_p_group_by_p_region_id_p_supplier_id_mixed
  • partiql_tests::eval::query::group_by::group_by::group_by::group_by_test::select_supplier_id_mixed_from_products_sparse_p_group_by_p_supplier_id_mixed
  • partiql_tests::eval::query::group_by::group_by::group_by::group_by_test::select_some_missing_from_simple_1_col_1_group_group_by_missing_as_some_missing
  • partiql_tests::eval::query::group_by::group_by::group_by::group_by_test::select_value_some_missing_some_missing_from_simple_1_col_1_group_group_by_missing_as_some_missing
  • partiql_tests::eval::query::group_by::group_by::group_by::group_by_test::select_p_region_id_p_supplier_id_missings_from_products_sparse_p_group_by_p_region_id_p_supplier_id_missings

Copy link
Contributor

@am357 am357 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM; left couple of minor comments and one question?

year: i32,
month: NonZeroU8,
day: u8,
hour: u8,
minute: u8,
second: f64,
second: u8,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-iterating clippy's warning regarding too many arguments; it's a warning so not a blocker but maybe we can move the args to a struct?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be fine to leave it as a warning for now. We might need to adjust some of these datetime functions when we sort out the date/time modeling in the conformance tests partiql/partiql-tests#82 and also support the date time literals.

partiql-eval/src/eval/expr/mod.rs Show resolved Hide resolved
partiql-logical-planner/src/call_defs.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@jpschorr jpschorr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My only comment was the extraneous % 60 in EvalFnExtractTimezoneMinute which was removed in the latest commit.

@alancai98 alancai98 merged commit e63267a into main Apr 17, 2023
@alancai98 alancai98 deleted the feat-extract branch April 17, 2023 23:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants