Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add YEARWEEK Function To OpenSearch SQL #236

Merged
5 changes: 5 additions & 0 deletions core/src/main/java/org/opensearch/sql/expression/DSL.java
Original file line number Diff line number Diff line change
Expand Up @@ -453,6 +453,11 @@ public static FunctionExpression year(Expression... expressions) {
return compile(FunctionProperties.None, BuiltinFunctionName.YEAR, expressions);
}

public static FunctionExpression yearweek(
FunctionProperties functionProperties, Expression... expressions) {
return compile(functionProperties, BuiltinFunctionName.YEARWEEK, expressions);
}

public static FunctionExpression divide(Expression... expressions) {
return compile(FunctionProperties.None, BuiltinFunctionName.DIVIDE, expressions);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,7 @@ public void register(BuiltinFunctionRepository repository) {
repository.register(week(BuiltinFunctionName.WEEKOFYEAR));
repository.register(week(BuiltinFunctionName.WEEK_OF_YEAR));
repository.register(year());
repository.register(yearweek());
}

/**
Expand Down Expand Up @@ -900,6 +901,30 @@ private DefaultFunctionResolver year() {
);
}

/**
* YEARWEEK(DATE[,mode]). return the week number for date.
*/
private DefaultFunctionResolver yearweek() {
return define(BuiltinFunctionName.YEARWEEK.getName(),
implWithProperties(nullMissingHandlingWithProperties((functionProperties, arg)
-> DateTimeFunction.yearweekToday(
DEFAULT_WEEK_OF_YEAR_MODE,
functionProperties.getQueryStartClock())), INTEGER, TIME),
impl(nullMissingHandling(DateTimeFunction::exprYearweekWithoutMode), INTEGER, DATE),
impl(nullMissingHandling(DateTimeFunction::exprYearweekWithoutMode), INTEGER, DATETIME),
impl(nullMissingHandling(DateTimeFunction::exprYearweekWithoutMode), INTEGER, TIMESTAMP),
impl(nullMissingHandling(DateTimeFunction::exprYearweekWithoutMode), INTEGER, STRING),
implWithProperties(nullMissingHandlingWithProperties((functionProperties, time, modeArg)
-> DateTimeFunction.yearweekToday(

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is DateTimeFunction. needed ..?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 271bf2d

modeArg,
functionProperties.getQueryStartClock())), INTEGER, TIME, INTEGER),
impl(nullMissingHandling(DateTimeFunction::exprYearweek), INTEGER, DATE, INTEGER),
impl(nullMissingHandling(DateTimeFunction::exprYearweek), INTEGER, DATETIME, INTEGER),
impl(nullMissingHandling(DateTimeFunction::exprYearweek), INTEGER, TIMESTAMP, INTEGER),
impl(nullMissingHandling(DateTimeFunction::exprYearweek), INTEGER, STRING, INTEGER)
);
}

/**
* Formats date according to format specifier. First argument is date, second is format.
* Detailed supported signatures:
Expand Down Expand Up @@ -1731,6 +1756,72 @@ private ExprValue exprYear(ExprValue date) {
return new ExprIntegerValue(date.dateValue().getYear());
}

/**
* Convert mode argument passed into our CalendarLookup class to a different mode.
* Needed to align with MySQL for the yearweek function due to different behaviour for modes.
* Note, this misalignment only exists for yearweek.
* Our current mode behavior works as intended for other functions.
*
* @param mode is an integer containing the initial mode arg
* @return an integer containing the new mode
*/
private int convertWeekModeFromMySqlToJava(LocalDate date, int mode) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you move this function under extractYearweek?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consider naming your parameter modeMySql to make it explicit. And returning modeJava in the comments.
Then you could call the function convertToWeekModeJava

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved under extractYearWeek c302e05

// Needed to align with MySQL. Due to how modes for this function work.
// See description of modes here ...
// https://dev.mysql.com/doc/refman/8.0/en/date-and-time-functions.html#function_week

if (CalendarLookup.getWeekNumber(mode, date) == 0) {
if (mode == 0 || mode == 1) {
mode = 2;
} else if (mode == 5) {
mode = 7;
}
Yury-Fridlyand marked this conversation as resolved.
Show resolved Hide resolved
}

return mode;
}

/**
* Helper function to extract the yearweek output from a given date.
*
* @param date is a LocalDate input argument.
* @param mode is an integer containing the mode used to parse the LocalDate.
* @return is a long containing the formatted output for the yearweek function.
*/
private int extractYearweek(LocalDate date, int mode) {
mode = convertWeekModeFromMySqlToJava(date, mode);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Create a new var called modeJava. It's best not to update parameters.
Call the input parameter to modeMySql to be explicit.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 271bf2d

int formatted = CalendarLookup.getYearNumber(mode, date) * 100
+ CalendarLookup.getWeekNumber(mode, date);

return formatted;
}

/**
* Yearweek for date implementation for ExprValue.
*
* @param date ExprValue of Date/Datetime/Time/Timestamp/String type.
* @param mode ExprValue of Integer type.
*/
private ExprValue exprYearweek(ExprValue date, ExprValue mode) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return ExprIntegerValue and you can avoid creating new ExprIntegerValue twice

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 271bf2d

return new ExprIntegerValue(extractYearweek(date.dateValue(), mode.integerValue()));
}

/**
* Yearweek for date implementation for ExprValue.
* When mode is not specified default value mode 0 is used.
*
* @param date ExprValue of Date/Datetime/Time/Timestamp/String type.
* @return ExprValue.
*/
private ExprValue exprYearweekWithoutMode(ExprValue date) {
return exprYearweek(date, new ExprIntegerValue(0));
Yury-Fridlyand marked this conversation as resolved.
Show resolved Hide resolved
}

private ExprValue yearweekToday(ExprValue mode, Clock clock) {
return new ExprIntegerValue(
extractYearweek(LocalDateTime.now(clock).toLocalDate(), mode.integerValue()));
}

private ExprValue monthOfYearToday(Clock clock) {
return new ExprIntegerValue(LocalDateTime.now(clock).getMonthValue());
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,7 @@ public enum BuiltinFunctionName {
WEEKOFYEAR(FunctionName.of("weekofyear")),
WEEK_OF_YEAR(FunctionName.of("week_of_year")),
YEAR(FunctionName.of("year")),
YEARWEEK(FunctionName.of("yearweek")),
// `now`-like functions
NOW(FunctionName.of("now")),
CURDATE(FunctionName.of("curdate")),
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/


package org.opensearch.sql.expression.datetime;

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.MethodSource;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.opensearch.sql.data.model.ExprDateValue;
import org.opensearch.sql.data.model.ExprTimeValue;
import org.opensearch.sql.data.model.ExprValue;
import org.opensearch.sql.exception.SemanticCheckException;
import org.opensearch.sql.expression.DSL;
import org.opensearch.sql.expression.Expression;
import org.opensearch.sql.expression.ExpressionTestBase;
import org.opensearch.sql.expression.FunctionExpression;
import org.opensearch.sql.expression.env.Environment;

import java.time.LocalDate;
import java.util.stream.Stream;

import static java.time.temporal.ChronoField.ALIGNED_WEEK_OF_YEAR;
Yury-Fridlyand marked this conversation as resolved.
Show resolved Hide resolved
import static org.junit.jupiter.api.Assertions.assertAll;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.opensearch.sql.data.model.ExprValueUtils.integerValue;
import static org.opensearch.sql.data.type.ExprCoreType.INTEGER;

@ExtendWith(MockitoExtension.class)
class YearweekTest extends ExpressionTestBase {

@Mock
Environment<Expression, ExprValue> env;

private void yearweekQuery(String date, int mode, int expectedResult) {
FunctionExpression expression = DSL
.yearweek(
functionProperties,
DSL.literal(new ExprDateValue(date)), DSL.literal(mode));
assertEquals(INTEGER, expression.type());
assertEquals(String.format("yearweek(DATE '%s', %d)", date, mode), expression.toString());
assertEquals(integerValue(expectedResult), eval(expression));
}

private static Stream<Arguments> getTestDataForYearweek() {
//Test the behavior of different modes passed into the 'yearweek' function
return Stream.of(
Arguments.of("2019-01-05", 0, 201852),
Arguments.of("2019-01-05", 1, 201901),
Arguments.of("2019-01-05", 2, 201852),
Arguments.of("2019-01-05", 3, 201901),
Arguments.of("2019-01-05", 4, 201901),
Arguments.of("2019-01-05", 5, 201853),
Arguments.of("2019-01-05", 6, 201901),
Arguments.of("2019-01-05", 7, 201853),
Arguments.of("2019-01-06", 0, 201901),
Arguments.of("2019-01-06", 1, 201901),
Arguments.of("2019-01-06", 2, 201901),
Arguments.of("2019-01-06", 3, 201901),
Arguments.of("2019-01-06", 4, 201902),
Arguments.of("2019-01-06", 5, 201853),
Arguments.of("2019-01-06", 6, 201902),
Arguments.of("2019-01-06", 7, 201853),
Arguments.of("2019-01-07", 0, 201901),
Arguments.of("2019-01-07", 1, 201902),
Arguments.of("2019-01-07", 2, 201901),
Arguments.of("2019-01-07", 3, 201902),
Arguments.of("2019-01-07", 4, 201902),
Arguments.of("2019-01-07", 5, 201901),
Arguments.of("2019-01-07", 6, 201902),
Arguments.of("2019-01-07", 7, 201901),
Arguments.of("2000-01-01", 0, 199952),
Arguments.of("2000-01-01", 2, 199952),
Arguments.of("1999-12-31", 0, 199952)
);
}

@ParameterizedTest(name = "{0} | {1}")
@MethodSource("getTestDataForYearweek")
public void testWeekyear(String date, int mode, int expected) {
yearweekQuery(date, mode, expected);
}

@Test
public void testYearWeekWithTimeType() {
int week = LocalDate.now(functionProperties.getQueryStartClock()).get(ALIGNED_WEEK_OF_YEAR);
int year = LocalDate.now(functionProperties.getQueryStartClock()).getYear();
int expected = Integer.parseInt(String.format("%d%02d", year, week));

FunctionExpression expression = DSL
.yearweek(
functionProperties,
DSL.literal(new ExprTimeValue("10:11:12")), DSL.literal(0));

assertEquals(expected, eval(expression).integerValue());
}

Yury-Fridlyand marked this conversation as resolved.
Show resolved Hide resolved
@Test
public void testInvalidYearWeek() {
assertAll(
//test invalid month
() -> assertThrows(
SemanticCheckException.class,
() -> yearweekQuery("2019-13-05 01:02:03", 0, 0)),
//test invalid day
() -> assertThrows(
SemanticCheckException.class,
() -> yearweekQuery("2019-01-50 01:02:03", 0, 0)),
//test invalid leap year
() -> assertThrows(
SemanticCheckException.class,
() -> yearweekQuery("2019-02-29 01:02:03", 0, 0))
);
}
private ExprValue eval(Expression expression) {
return expression.valueOf(env);
Yury-Fridlyand marked this conversation as resolved.
Show resolved Hide resolved
}
}
22 changes: 22 additions & 0 deletions docs/user/dql/functions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2698,6 +2698,28 @@ Example::
+----------------------------+


YEARWEEK
--------

Description
>>>>>>>>>>>

Usage: yearweek(date) returns the year and week for date as an integer. It accepts and optional mode arguments aligned with those available for the `WEEK`_ function.

Argument type: STRING/DATE/DATETIME/TIME/TIMESTAMP

Return type: INTEGER

Example::
Yury-Fridlyand marked this conversation as resolved.
Show resolved Hide resolved

os> SELECT YEARWEEK('2020-08-26'), YEARWEEK('2019-01-05', 0),
fetched rows / total rows = 1/1
+--------------------------+-----------------------------+
| YEARWEEK('2020-08-26') | YEARWEEK('2019-01-05', 0) |
|--------------------------+-----------------------------+
| 202034 | 201852 |
+--------------------------+-----------------------------+

String Functions
================

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -985,6 +985,14 @@ public void testWeekAlternateSyntaxesReturnTheSameResults() throws IOException {
compareWeekResults("CAST(datetime0 AS timestamp)", TEST_INDEX_CALCS);
}

@Test
public void testYearweek() throws IOException {
JSONObject result = executeQuery(
String.format("SELECT yearweek(time0), yearweek(time0, 4) FROM %s LIMIT 2", TEST_INDEX_CALCS));

verifyDataRows(result, rows(189952, 189952), rows(189953, 190001));
}

void verifyDateFormat(String date, String type, String format, String formatted) throws IOException {
String query = String.format("date_format(%s('%s'), '%s')", type, date, format);
JSONObject result = executeQuery("select " + query);
Expand Down
1 change: 1 addition & 0 deletions sql/src/main/antlr/OpenSearchSQLLexer.g4
Original file line number Diff line number Diff line change
Expand Up @@ -340,6 +340,7 @@ STRCMP: 'STRCMP';

// DATE AND TIME FUNCTIONS
ADDDATE: 'ADDDATE';
YEARWEEK: 'YEARWEEK';

// RELEVANCE FUNCTIONS AND PARAMETERS
ALLOW_LEADING_WILDCARD: 'ALLOW_LEADING_WILDCARD';
Expand Down
1 change: 1 addition & 0 deletions sql/src/main/antlr/OpenSearchSQLParser.g4
Original file line number Diff line number Diff line change
Expand Up @@ -482,6 +482,7 @@ dateTimeFunctionName
| WEEK_OF_YEAR
| WEEKOFYEAR
| YEAR
| YEARWEEK
;

textFunctionName
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -542,6 +542,12 @@ public void can_parse_wildcard_query_relevance_function() {
+ "boost=1.5, case_insensitive=true, rewrite=\"scoring_boolean\")"));
}

@Test
public void can_parse_yearweek_function() {
assertNotNull(parser.parse("SELECT yearweek('1987-01-01')"));
assertNotNull(parser.parse("SELECT yearweek('1987-01-01', 1)"));
}

@Test
public void describe_request_accepts_only_quoted_string_literals() {
assertAll(
Expand Down