Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement datetime functions in velox/sparksql. #81

Merged
merged 2 commits into from
Nov 28, 2022
Merged

Implement datetime functions in velox/sparksql. #81

merged 2 commits into from
Nov 28, 2022

Conversation

lviiii
Copy link

@lviiii lviiii commented Nov 23, 2022

No description provided.

@lviiii lviiii mentioned this pull request Nov 28, 2022
Merged
rui-mo
rui-mo reviewed Nov 28, 2022
View reviewed changes
velox/functions/sparksql/Register.cpp
registerFunction<YearOfWeekFunction, int32_t, Timestamp>(
{prefix + "year_of_week"});
registerFunction<YearOfWeekFunction, int32_t, TimestampWithTimezone>(
{prefix + "year_of_week"});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you going to open PR to Velox to add these functions to spark sql?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I've opened this PR(facebookincubator#3362)

rui-mo
rui-mo approved these changes Nov 28, 2022
View reviewed changes
Copy link
Collaborator

@rui-mo rui-mo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@rui-mo rui-mo merged commit eef5982 into oap-project:main Nov 28, 2022
@lviiii lviiii deleted the velox_datetime_func branch December 2, 2022 11:22
zhejiangxiaomai pushed a commit to zhejiangxiaomai/velox that referenced this pull request Dec 12, 2022
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (oap-project#81)
2de0512
zhejiangxiaomai pushed a commit to zhejiangxiaomai/velox that referenced this pull request Dec 13, 2022
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (oap-project#81)
d6034ba
zhejiangxiaomai pushed a commit to zhejiangxiaomai/velox that referenced this pull request Dec 14, 2022
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (oap-project#81)
8c5f9e2
zhejiangxiaomai pushed a commit to zhejiangxiaomai/velox that referenced this pull request Dec 14, 2022
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (oap-project#81)
2cc6896
zhejiangxiaomai pushed a commit to zhejiangxiaomai/velox that referenced this pull request Dec 15, 2022
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (oap-project#81)
c02106a
zhejiangxiaomai pushed a commit to zhejiangxiaomai/velox that referenced this pull request Dec 15, 2022
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (oap-project#81)
b0e9e2d
zhejiangxiaomai pushed a commit to zhejiangxiaomai/velox that referenced this pull request Dec 21, 2022
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (oap-project#81)
d1b8280
zhejiangxiaomai pushed a commit to zhejiangxiaomai/velox that referenced this pull request Jan 3, 2023
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (oap-project#81)
4c98b1c
zhejiangxiaomai pushed a commit that referenced this pull request Jan 11, 2023
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (#81)
bda3cad
zhejiangxiaomai pushed a commit that referenced this pull request Jan 11, 2023
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (#81)
7512423
zhejiangxiaomai pushed a commit that referenced this pull request Jan 31, 2023
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (#81)
0645968
zhejiangxiaomai pushed a commit that referenced this pull request Feb 22, 2023
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (#81)
dcd79e7
zhejiangxiaomai pushed a commit to zhejiangxiaomai/velox that referenced this pull request Feb 27, 2023
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (oap-project#81)
a11cc29
zhejiangxiaomai pushed a commit to zhejiangxiaomai/velox that referenced this pull request Mar 6, 2023
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (oap-project#81)
6557134
zhejiangxiaomai pushed a commit to zhejiangxiaomai/velox that referenced this pull request Mar 27, 2023
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (oap-project#81)
b6001a8
zhejiangxiaomai pushed a commit to zhejiangxiaomai/velox that referenced this pull request Mar 29, 2023
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (oap-project#81)
a4490bb
zhejiangxiaomai pushed a commit to zhejiangxiaomai/velox that referenced this pull request Mar 29, 2023
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (oap-project#81)
a1a445b
zhejiangxiaomai pushed a commit to zhejiangxiaomai/velox that referenced this pull request Apr 14, 2023
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (oap-project#81)
f3ed02e
zhejiangxiaomai pushed a commit to zhejiangxiaomai/velox that referenced this pull request Apr 17, 2023
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (oap-project#81)
6d06729
zhejiangxiaomai pushed a commit to zhejiangxiaomai/velox that referenced this pull request Apr 19, 2023
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (oap-project#81)
46073b6
zhejiangxiaomai pushed a commit to zhejiangxiaomai/velox that referenced this pull request Apr 20, 2023
@lviiii @zhejiangxiaomai
Implement datetime functions in velox/sparksql. (oap-project#81)
78d2a4c
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request May 31, 2023
@zhejiangxiaomai
Folder: function
1bbd42b 
relative pr:

Fix replace SparkSQL function oap-project#277
Support kPreceeding & kFollowing for window range frame type oap-project#287
support timestamp hash oap-project#269
Spark sum can overflow oap-project#101
Support float & double types in pmod function oap-project#157
Implement datetime functions in velox/sparksql. oap-project#81
Fix type check in MapFunction oap-project#273
Let function validation fail for lookaround pattern in RE2-based implementation oap-project#124
Register lpad/rpad functions for Spark SQL. oap-project#63
Support substring_index sql function oap-project#189
Fix First/Last aggregate functions intermediate type and support decimal oap-project#245
Support date_add spark sql function oap-project#144
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request May 31, 2023
@zhejiangxiaomai
Folder: function
731b4be 
relative pr:

Fix replace SparkSQL function oap-project#277
Support kPreceeding & kFollowing for window range frame type oap-project#287
support timestamp hash oap-project#269
Spark sum can overflow oap-project#101
Support float & double types in pmod function oap-project#157
Implement datetime functions in velox/sparksql. oap-project#81
Fix type check in MapFunction oap-project#273
Let function validation fail for lookaround pattern in RE2-based implementation oap-project#124
Register lpad/rpad functions for Spark SQL. oap-project#63
Support substring_index sql function oap-project#189
Fix First/Last aggregate functions intermediate type and support decimal oap-project#245
Support date_add spark sql function oap-project#144
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request May 31, 2023
@zhejiangxiaomai
Folder: substrait
efc0949 
relative pr:

Check a fallback case in validation: using literal partition key in window function oap-project#148
Fix might_contain validate fallback and support struct literal oap-project#137
Implement datetime functions in velox/sparksql. oap-project#81
Parse options in SingularOrList correctly oap-project#48
Add SingularOrList support oap-project#45
Support if then in filter oap-project#74
Fix semi join output type and support existence join oap-project#67
Support decimal as partition column oap-project#167
Add the window support oap-project#61
Add expand operator oap-project#65
Support more cases of filter and its pushdown #14
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request May 31, 2023
@zhejiangxiaomai
Folder: substrait
d8ad570 
relative pr:

Check a fallback case in validation: using literal partition key in window function oap-project#148
Fix might_contain validate fallback and support struct literal oap-project#137
Implement datetime functions in velox/sparksql. oap-project#81
Parse options in SingularOrList correctly oap-project#48
Add SingularOrList support oap-project#45
Support if then in filter oap-project#74
Fix semi join output type and support existence join oap-project#67
Support decimal as partition column oap-project#167
Add the window support oap-project#61
Add expand operator oap-project#65
Support more cases of filter and its pushdown #14
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request May 31, 2023
@zhejiangxiaomai
Folder: substrait
f5e31fa 
relative pr:

Check a fallback case in validation: using literal partition key in window function oap-project#148
Fix might_contain validate fallback and support struct literal oap-project#137
Implement datetime functions in velox/sparksql. oap-project#81
Parse options in SingularOrList correctly oap-project#48
Add SingularOrList support oap-project#45
Support if then in filter oap-project#74
Fix semi join output type and support existence join oap-project#67
Support decimal as partition column oap-project#167
Add the window support oap-project#61
Add expand operator oap-project#65
Support more cases of filter and its pushdown #14
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request Jul 3, 2023
@zhejiangxiaomai
Folder: function
44b2919 
relative pr:

Fix replace SparkSQL function oap-project#277
Support kPreceeding & kFollowing for window range frame type oap-project#287
support timestamp hash oap-project#269
Spark sum can overflow oap-project#101
Support float & double types in pmod function oap-project#157
Implement datetime functions in velox/sparksql. oap-project#81
Fix type check in MapFunction oap-project#273
Let function validation fail for lookaround pattern in RE2-based implementation oap-project#124
Register lpad/rpad functions for Spark SQL. oap-project#63
Support substring_index sql function oap-project#189
Fix First/Last aggregate functions intermediate type and support decimal oap-project#245
Support date_add spark sql function oap-project#144
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request Jul 3, 2023
@zhejiangxiaomai
Folder: substrait
e94db80 
relative pr:

Check a fallback case in validation: using literal partition key in window function oap-project#148
Fix might_contain validate fallback and support struct literal oap-project#137
Implement datetime functions in velox/sparksql. oap-project#81
Parse options in SingularOrList correctly oap-project#48
Add SingularOrList support oap-project#45
Support if then in filter oap-project#74
Fix semi join output type and support existence join oap-project#67
Support decimal as partition column oap-project#167
Add the window support oap-project#61
Add expand operator oap-project#65
Support more cases of filter and its pushdown #14
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request Jul 4, 2023
@zhejiangxiaomai
Folder: function
eaf2b52 
relative pr:

Fix replace SparkSQL function oap-project#277
Support kPreceeding & kFollowing for window range frame type oap-project#287
support timestamp hash oap-project#269
Spark sum can overflow oap-project#101
Support float & double types in pmod function oap-project#157
Implement datetime functions in velox/sparksql. oap-project#81
Fix type check in MapFunction oap-project#273
Let function validation fail for lookaround pattern in RE2-based implementation oap-project#124
Register lpad/rpad functions for Spark SQL. oap-project#63
Support substring_index sql function oap-project#189
Fix First/Last aggregate functions intermediate type and support decimal oap-project#245
Support date_add spark sql function oap-project#144
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request Jul 4, 2023
@zhejiangxiaomai
Folder: substrait
6ae5844 
relative pr:

Check a fallback case in validation: using literal partition key in window function oap-project#148
Fix might_contain validate fallback and support struct literal oap-project#137
Implement datetime functions in velox/sparksql. oap-project#81
Parse options in SingularOrList correctly oap-project#48
Add SingularOrList support oap-project#45
Support if then in filter oap-project#74
Fix semi join output type and support existence join oap-project#67
Support decimal as partition column oap-project#167
Add the window support oap-project#61
Add expand operator oap-project#65
Support more cases of filter and its pushdown #14
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request Jul 11, 2023
@zhejiangxiaomai
Folder: function
4de772f 
relative pr:

Fix replace SparkSQL function oap-project#277
Support kPreceeding & kFollowing for window range frame type oap-project#287
support timestamp hash oap-project#269
Spark sum can overflow oap-project#101
Support float & double types in pmod function oap-project#157
Implement datetime functions in velox/sparksql. oap-project#81
Fix type check in MapFunction oap-project#273
Let function validation fail for lookaround pattern in RE2-based implementation oap-project#124
Register lpad/rpad functions for Spark SQL. oap-project#63
Support substring_index sql function oap-project#189
Fix First/Last aggregate functions intermediate type and support decimal oap-project#245
Support date_add spark sql function oap-project#144
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request Jul 11, 2023
@zhejiangxiaomai
Folder: substrait
7efb94f 
relative pr:

Check a fallback case in validation: using literal partition key in window function oap-project#148
Fix might_contain validate fallback and support struct literal oap-project#137
Implement datetime functions in velox/sparksql. oap-project#81
Parse options in SingularOrList correctly oap-project#48
Add SingularOrList support oap-project#45
Support if then in filter oap-project#74
Fix semi join output type and support existence join oap-project#67
Support decimal as partition column oap-project#167
Add the window support oap-project#61
Add expand operator oap-project#65
Support more cases of filter and its pushdown #14
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request Jul 12, 2023
@zhejiangxiaomai
Folder: function
e8d7b27 
relative pr:

Fix replace SparkSQL function oap-project#277
Support kPreceeding & kFollowing for window range frame type oap-project#287
support timestamp hash oap-project#269
Spark sum can overflow oap-project#101
Support float & double types in pmod function oap-project#157
Implement datetime functions in velox/sparksql. oap-project#81
Fix type check in MapFunction oap-project#273
Let function validation fail for lookaround pattern in RE2-based implementation oap-project#124
Register lpad/rpad functions for Spark SQL. oap-project#63
Support substring_index sql function oap-project#189
Fix First/Last aggregate functions intermediate type and support decimal oap-project#245
Support date_add spark sql function oap-project#144
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request Jul 12, 2023
@zhejiangxiaomai
Folder: substrait
7bbccfb 
relative pr:

Check a fallback case in validation: using literal partition key in window function oap-project#148
Fix might_contain validate fallback and support struct literal oap-project#137
Implement datetime functions in velox/sparksql. oap-project#81
Parse options in SingularOrList correctly oap-project#48
Add SingularOrList support oap-project#45
Support if then in filter oap-project#74
Fix semi join output type and support existence join oap-project#67
Support decimal as partition column oap-project#167
Add the window support oap-project#61
Add expand operator oap-project#65
Support more cases of filter and its pushdown #14
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request Jul 12, 2023
@zhejiangxiaomai
Folder: function
20e47f5 
relative pr:

Fix replace SparkSQL function oap-project#277
Support kPreceeding & kFollowing for window range frame type oap-project#287
support timestamp hash oap-project#269
Spark sum can overflow oap-project#101
Support float & double types in pmod function oap-project#157
Implement datetime functions in velox/sparksql. oap-project#81
Fix type check in MapFunction oap-project#273
Let function validation fail for lookaround pattern in RE2-based implementation oap-project#124
Register lpad/rpad functions for Spark SQL. oap-project#63
Support substring_index sql function oap-project#189
Fix First/Last aggregate functions intermediate type and support decimal oap-project#245
Support date_add spark sql function oap-project#144
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request Jul 12, 2023
@zhejiangxiaomai
Folder: substrait
e93466c 
relative pr:

Check a fallback case in validation: using literal partition key in window function oap-project#148
Fix might_contain validate fallback and support struct literal oap-project#137
Implement datetime functions in velox/sparksql. oap-project#81
Parse options in SingularOrList correctly oap-project#48
Add SingularOrList support oap-project#45
Support if then in filter oap-project#74
Fix semi join output type and support existence join oap-project#67
Support decimal as partition column oap-project#167
Add the window support oap-project#61
Add expand operator oap-project#65
Support more cases of filter and its pushdown #14
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request Jul 17, 2023
@zhejiangxiaomai
Folder: function
e7fa7f5 
relative pr:

Fix replace SparkSQL function oap-project#277
Support kPreceeding & kFollowing for window range frame type oap-project#287
support timestamp hash oap-project#269
Spark sum can overflow oap-project#101
Support float & double types in pmod function oap-project#157
Implement datetime functions in velox/sparksql. oap-project#81
Fix type check in MapFunction oap-project#273
Let function validation fail for lookaround pattern in RE2-based implementation oap-project#124
Register lpad/rpad functions for Spark SQL. oap-project#63
Support substring_index sql function oap-project#189
Fix First/Last aggregate functions intermediate type and support decimal oap-project#245
Support date_add spark sql function oap-project#144
zhejiangxiaomai added a commit to zhejiangxiaomai/velox that referenced this pull request Jul 17, 2023
@zhejiangxiaomai
Folder: substrait
20f54b6 
relative pr:

Check a fallback case in validation: using literal partition key in window function oap-project#148
Fix might_contain validate fallback and support struct literal oap-project#137
Implement datetime functions in velox/sparksql. oap-project#81
Parse options in SingularOrList correctly oap-project#48
Add SingularOrList support oap-project#45
Support if then in filter oap-project#74
Fix semi join output type and support existence join oap-project#67
Support decimal as partition column oap-project#167
Add the window support oap-project#61
Add expand operator oap-project#65
Support more cases of filter and its pushdown #14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rui-mo rui-mo rui-mo approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@lviiii @rui-mo