-
Notifications
You must be signed in to change notification settings - Fork 141
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding CONVERT_TZ
and DATETIME
functions to SQL and PPL
#848
Merged
dai-chen
merged 61 commits into
opensearch-project:2.x
from
Bit-Quill:integ-Add-convert_tz-function
Sep 28, 2022
Merged
Adding CONVERT_TZ
and DATETIME
functions to SQL and PPL
#848
dai-chen
merged 61 commits into
opensearch-project:2.x
from
Bit-Quill:integ-Add-convert_tz-function
Sep 28, 2022
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
87 tasks
Yury-Fridlyand
changed the title
Adding convert_tz and datetime functions to SQL and PPL
Adding Sep 20, 2022
CONVERT_TZ
and DATETIME
functions to SQL and PPL
dai-chen
reviewed
Sep 21, 2022
core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java
Outdated
Show resolved
Hide resolved
core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java
Outdated
Show resolved
Hide resolved
Could you point me to the documentation for |
dai-chen
added
enhancement
New feature or request
SQL
PPL
Piped processing language
labels
Sep 21, 2022
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
…esolver convert_tz and ExprValue exprConvert_TZ. It implements the functionality for converting between time zones. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
…s that are outside the existing range (consistent with MySQL standard). Added implementation for convert_tz consistent with MySQL implementation. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
…nctionTest.java. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
…unction call conevrt_tz function. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
MitchellGale
force-pushed
the
integ-Add-convert_tz-function
branch
from
September 28, 2022 19:10
f19780b
to
eb0c894
Compare
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
MitchellGale
force-pushed
the
integ-Add-convert_tz-function
branch
from
September 28, 2022 19:22
b6011ff
to
3d64678
Compare
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Yury-Fridlyand
approved these changes
Sep 28, 2022
acarbonetto
approved these changes
Sep 28, 2022
dai-chen
approved these changes
Sep 28, 2022
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Thanks for the changes!
MitchellGale
added a commit
to Bit-Quill/opensearch-project-sql
that referenced
this pull request
Oct 3, 2022
…ch-project#848) * Added `CONVERT_TZ` to the PPL lexer/parser and the SQL lexer/parser. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Added `convert_tz` to the BuiltinFunctionName.java. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Added convert_tz to DateTimeFunction.java register, private FunctionResolver convert_tz and ExprValue exprConvert_TZ. It implements the functionality for converting between time zones. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Added IT PPL and SQL tests for various conditions including time zones that are outside the existing range (consistent with MySQL standard). Added implementation for convert_tz consistent with MySQL implementation. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Added DATETIME to OpenSearchPPLParser.g4 and OpenSearchSQLParser.g4. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Rebase merge conflict resolution. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Added ppl doctest for convert_tz Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Completed implementation for datetime and convert_Tz. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Removed SQL test from PPL IT test Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Removed redundant convert to string. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Fixed doctests and reverted changes to adddate function in DateTimeFunctionTest.java. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Fixed doctest. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Fixed doctest. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Added additional integration tests for PPL and SQL tests for convert_tz. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Added null for timezones outside of basic range for DATETIME. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Added test cases for null with the datetime function. Made DateTime function call conevrt_tz function. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Seperated out test from DateTimeFunction.java. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Updated tests. Fixed exception to be less general. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Removed changes. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Fixed rel timezone issue. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Added support for non valid datetime to return null for convert_tz. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Made exception more verbose. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Removed unneeded format changes in DateTimeFunction.java. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Added more doctests. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Removed formatting changes. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Reverting sql/ppl DateTimeFunctionsIT.java. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Reverted changes to DateTimeFunctionTest.java Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Added more information about invalid date for convert_tz Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Converted "from Field" and "To Field" to use "Fieldn" where n is the field number. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Added date validation. Added test cases in IT to cover cases. Added test in ConvertTZTest.java. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Fixed date validation function. Broke up some unit tests. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Fixed formatting. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Added DateTime tests, broke up functions. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Added space in DateTimeTest.java. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Tidied up code and tests. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Fixed local date time rst test. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Removed nested try/catch exceptions. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * removed exprConvertTZ function call from within try catch statement. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Reverted change from parse localdate Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Removed extra casting around fromTz variable. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Updated wording for functions.rst Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Updated wording for datetime.rst to describe null for conert_tz Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Added more test cases for functions.rst Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Fixed doctests. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Removed extra import. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Renamed isValidTimeZone function to isValidMySqlTimeZoneId. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Has ExprDatetimeValue doing work for exprConvertTZ call. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Deleted fromTZ Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Moved fixed variables to top of class. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Added Null to support of exprDateTime. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Added missing expr functions for makedate/time Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * cleaning up after rebase merge. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Rebase merge conflict resolution. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Fixed missing DATETIME in SQL Parser. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Fixed IT test. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Addressed PR comments Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Added missing variables after rebase Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Fixed checkstyle errors after rebase. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Moved formatter for date time over. Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Changed function resolved to default Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> * Removed unneeded code Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com> Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
PR adds support for convert_tz and Datetime for PPL and SQL languages.
convert_tz
Syntax for convert_tz:
convert_tz(datetime string, from timezone, to timezone)
It converts the
datetime string
from thefrom timezone
to theto timezone
. Converts from negative to positive timezone, from positive to negative, from positive to positive or negative to negative time zones. Works across year roll-over and leap years.Example:
nulls
It returns null for time zones outside of the allowed range (allow range is -13:59 -> +14:00)
Leap year
DateTime
DATETIME(timestamp_expression [, time_zone])
Issues Resolved
#703
#46
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.