Skip to content

Commit

Permalink
Implemented PS-5764 (Introduce SEQUENCE_TABLE() table-level SQL funct…
Browse files Browse the repository at this point in the history
…ion)

https://jira.percona.com/browse/PS-5764

Implemented 'SEQUENCE_TABLE(<expr>)' table function that is supposed to be
used for generating sequences of integers. For instance

'SELECT * FROM SEQUENCE_TABLE(3) AS tt'
should return
+-------+
| value |
+-------+
| 0     |
| 1     |
| 2     |
+-------+
Please notice that as for other table functions (like 'JSON_TABLE()') table
alias ('AS tt' in the example) is not optional.

The result set will include a single column named 'value' that will be defined
as 'BIGINT UNSIGNED NOT NULL'. In other words, the following two statements
'CREATE TABLE t1 AS SELECT * FROM SEQUENCE_TABLE(...) AS tt;'
'SHOW CREATE TABLE t1;'
will result in
"Table Create Table"
"t1 CREATE TABLE `t1` ("
"  `value` bigint unsigned NOT NULL DEFAULT '0'"
") ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci".

Introduced new global dynamic variable 'tf_sequence_table_max_upper_bound'
that determines max value of how many records 'SEQUENCE_TABLE()' table
function is allowed to generate. Default - 1048576, min - 1024,
max - 2^64 - 1.

The max value of the expression passed to 'SEQUENCE_TABLE()' should not
exceed '@@tf_sequence_table_max_upper_bound'. Otherwise,
'ER_SEQUENCE_TABLE_SIZE_LIMIT' error will be reported.

Similarly to 'Table_function_json' class that is used for 'JSON_TABLE()',
introduced a new 'Table_function_sequence' class derived from 'Table_function'
that is supposed to fill virtual result table with a zero-based sequence of
integers.

Depending on the type of the '<expr>' passed to 'SEQUENCE_TABLE(<expr>)' there
are three possible cases of its const-ness:

1. Always const (e.g. a numeric literal or an expression involving only
   numeric literals), '<expr>' will be evaluated only once inside
   'Table_function_sequence::do_init_args()'.
2. Non-const during init(), but becomes const after it (e.g a field from a
   const table), '<expr>' will be evaluated only the first time
   'Table_function_sequence::fill_result_table()' is called. In every
   subsequent call of this function a precalculated value from the very first
   call will be re-used.
3. Non-const (e.g. a table field) '<expr>' will be evaluated every time
   'Table_function_sequence::fill_result_table()' is called.

Similarly to 'PT_table_factor_function' class that is used for 'JSON_TABLE()'
introduced a new 'PT_table_sequence_function' class derived from
'PT_table_reference' that is supposed to add a new virtual joined table into
'Parse_context' whose functionality implemented in 'Table_function_json'.

MySQL grammar extended with a alternative for 'table_function' - now there are
two 'JSON_TABLE_SYM' ... and 'SEQUENCE_TABLE_SYM' ...

Added 'main.percona_sequence_table' MTR test case that checks
'SEQUENCE_TABLE()' functionality in various SQL contexts:
- Checking invalid SEQUENCE_TABLE() arguments
- Checking SEQUENCE_TABLE() without table alias
- Simple usage scenarios
- Checking SEQUENCE_TABLE() max records limit
- Checking SEQUENCE_TABLE() upper bound value conversions (NULLs, negatives, floating points, strings)
- Checking SEQUENCE_TABLE() upper bound expressions (session variables, simple arithmetic)
- Checking SEQUENCE_TABLE() with WHERE clause (basic point selection, ranges, every second)
- Selecting custom value expressions from SEQUENCE_TABLE()
- Checking VIEWs based on SEQUENCE_TABLE()
- Checking stored procedures / functions
- Checking prepared statements
- Checking SHOW CREATE TABLE
- Checking table joins
- Checking for various forbidden upper_bound constructs

Added 'sys_vars.tf_sequence_table_max_upper_bound_basic' MTR test case that
checks basic variable functionality.

--

PS-9218: Merge MySQL 8.4.0 (fix percona.sequence_table) (percona#5331)

https://perconadev.atlassian.net/browse/PS-9218

Re-recorded 'percona.sequence_table' MTR test case because of the
WL #14061 "Access paths for iterators"
(commit mysql/mysql-server@55b9507).

--

PS-8963 fix: SEQUENCE_TABLE Issue

https://perconadev.atlassian.net/browse/PS-8963

Introduced new 'PERCONA_SEQUENCE_TABLE' reserved keyword which is supposed
to be a safe replacement for 'SEQUENCE_TABLE' which is now considered to
be deprecated.
Currently both have identical behavior except the fact that the latter now generates
a deprecation warning.

As an intermediate step before deprecated 'SEQUENCE_TABLE' is removed the
MySQL parser has been modified so that 'SEQUENCE_TABLE' is no longer a
reserved keyword and can now be used as a regular MySQL identifier (table name,
column name, alias name, etc).

'SEQUENCE_TABLE' token added to the list of the non-reserved keywords
('<ident_keywords_unambiguous>') in the Percona Server grammar file
('sql_yacc.yy').

Added new 'main.percona_sequence_table_keyword' MTR test case that check
various scenarios of 'SEQUENCE_TABLE' used as a simple table name, as an alias,
or a table function.

Re-recorded 'main.information_schema_keywords' MTR test case because
'SEQUENCE_TABLE' is no longer a reserved keyword.

Updated a number of existing MTR test cases with new 'PERCONA_SEQUENCE_TABLE()'
table-level function instead of 'SEQUENCE_TABLE()'.
  • Loading branch information
percona-ysorokin authored and dlenev committed Aug 28, 2024
1 parent 5b8d991 commit e23e6c3
Show file tree
Hide file tree
Showing 30 changed files with 1,937 additions and 12 deletions.
2 changes: 2 additions & 0 deletions mysql-test/r/information_schema_keywords.result
Original file line number Diff line number Diff line change
Expand Up @@ -435,6 +435,7 @@ PASSWORD 0
PASSWORD_LOCK_TIME 0
PATH 0
PERCENT_RANK 1
PERCONA_SEQUENCE_TABLE 1
PERSIST 0
PERSIST_ONLY 0
PHASE 0
Expand Down Expand Up @@ -554,6 +555,7 @@ SECURITY 0
SELECT 1
SENSITIVE 1
SEPARATOR 1
SEQUENCE_TABLE 0
SERIAL 0
SERIALIZABLE 0
SERVER 0
Expand Down
4 changes: 4 additions & 0 deletions mysql-test/r/mysqld--help-notwin.result
Original file line number Diff line number Diff line change
Expand Up @@ -1612,6 +1612,9 @@ The following options may be given as the first argument:
a version lower than 8.2.0. SHOW EVENTS and queries into
information_schema.events will also output the old
terminology for the event status field.
--tf-sequence-table-max-upper-bound=#
Maximum number of records PERCONA_SEQUENCE_TABLE() table
function is allowed to generate.
--thread-cache-size=#
How many threads we should keep in a cache for reuse
--thread-handling=name
Expand Down Expand Up @@ -2115,6 +2118,7 @@ tc-heuristic-recover OFF
temptable-max-mmap 0
temptable-use-mmap FALSE
terminology-use-previous NONE
tf-sequence-table-max-upper-bound 1048576
thread-cache-size 9
thread-handling one-thread-per-connection
thread-stack 1048576
Expand Down
2 changes: 1 addition & 1 deletion mysql-test/suite/federated/r/federated_debug.result
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ SELECT * FROM test.t1;
id
SET @debug_save= @@GLOBAL.DEBUG;
SET @@GLOBAL.DEBUG='+d,PS-8747_wait_for_disconnect_after_check';
INSERT INTO test.t1 SELECT tt.* FROM SEQUENCE_TABLE(20000) AS tt;
INSERT INTO test.t1 SELECT tt.* FROM PERCONA_SEQUENCE_TABLE(20000) AS tt;
SET GLOBAL DEBUG= @debug_save;
DROP TABLE test.t1;
DROP SERVER test;
Expand Down
2 changes: 1 addition & 1 deletion mysql-test/suite/federated/t/federated_debug.test
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ SET @@GLOBAL.DEBUG='+d,PS-8747_wait_for_disconnect_after_check';

# Send data which will not fit into one communication packet, so client will try to send them
# before flush and reconnection.
INSERT INTO test.t1 SELECT tt.* FROM SEQUENCE_TABLE(20000) AS tt;
INSERT INTO test.t1 SELECT tt.* FROM PERCONA_SEQUENCE_TABLE(20000) AS tt;
SET GLOBAL DEBUG= @debug_save;

DROP TABLE test.t1;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ OPTIMIZE TABLE t1;
--echo *** creating a list of all printable characters (ASCII 33..126)
--echo *** (whitespace and control characters are excluded)

SELECT GROUP_CONCAT(CHAR(value + 33 USING utf8mb4) SEPARATOR '') INTO @special_characters FROM SEQUENCE_TABLE(127 - 33) AS tt;
SELECT GROUP_CONCAT(CHAR(value + 33 USING utf8mb4) SEPARATOR '') INTO @special_characters FROM PERCONA_SEQUENCE_TABLE(127 - 33) AS tt;
SELECT @special_characters;

--echo
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ test.t1 optimize status OK

*** creating a list of all printable characters (ASCII 33..126)
*** (whitespace and control characters are excluded)
SELECT GROUP_CONCAT(CHAR(value + 33 USING utf8mb4) SEPARATOR '') INTO @special_characters FROM SEQUENCE_TABLE(127 - 33) AS tt;
SELECT GROUP_CONCAT(CHAR(value + 33 USING utf8mb4) SEPARATOR '') INTO @special_characters FROM PERCONA_SEQUENCE_TABLE(127 - 33) AS tt;
SELECT @special_characters;
@special_characters
!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ test.t1 optimize status OK

*** creating a list of all printable characters (ASCII 33..126)
*** (whitespace and control characters are excluded)
SELECT GROUP_CONCAT(CHAR(value + 33 USING utf8mb4) SEPARATOR '') INTO @special_characters FROM SEQUENCE_TABLE(127 - 33) AS tt;
SELECT GROUP_CONCAT(CHAR(value + 33 USING utf8mb4) SEPARATOR '') INTO @special_characters FROM PERCONA_SEQUENCE_TABLE(127 - 33) AS tt;
SELECT @special_characters;
@special_characters
!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ test.t1 optimize status OK

*** creating a list of all printable characters (ASCII 33..126)
*** (whitespace and control characters are excluded)
SELECT GROUP_CONCAT(CHAR(value + 33 USING utf8mb4) SEPARATOR '') INTO @special_characters FROM SEQUENCE_TABLE(127 - 33) AS tt;
SELECT GROUP_CONCAT(CHAR(value + 33 USING utf8mb4) SEPARATOR '') INTO @special_characters FROM PERCONA_SEQUENCE_TABLE(127 - 33) AS tt;
SELECT @special_characters;
@special_characters
!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ test.t1 optimize status OK

*** creating a list of all printable characters (ASCII 33..126)
*** (whitespace and control characters are excluded)
SELECT GROUP_CONCAT(CHAR(value + 33 USING utf8mb4) SEPARATOR '') INTO @special_characters FROM SEQUENCE_TABLE(127 - 33) AS tt;
SELECT GROUP_CONCAT(CHAR(value + 33 USING utf8mb4) SEPARATOR '') INTO @special_characters FROM PERCONA_SEQUENCE_TABLE(127 - 33) AS tt;
SELECT @special_characters;
@special_characters
!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ test.t1 optimize status OK

*** creating a list of all printable characters (ASCII 33..126)
*** (whitespace and control characters are excluded)
SELECT GROUP_CONCAT(CHAR(value + 33 USING utf8mb4) SEPARATOR '') INTO @special_characters FROM SEQUENCE_TABLE(127 - 33) AS tt;
SELECT GROUP_CONCAT(CHAR(value + 33 USING utf8mb4) SEPARATOR '') INTO @special_characters FROM PERCONA_SEQUENCE_TABLE(127 - 33) AS tt;
SELECT @special_characters;
@special_characters
!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ test.t1 optimize status OK

*** creating a list of all printable characters (ASCII 33..126)
*** (whitespace and control characters are excluded)
SELECT GROUP_CONCAT(CHAR(value + 33 USING utf8mb4) SEPARATOR '') INTO @special_characters FROM SEQUENCE_TABLE(127 - 33) AS tt;
SELECT GROUP_CONCAT(CHAR(value + 33 USING utf8mb4) SEPARATOR '') INTO @special_characters FROM PERCONA_SEQUENCE_TABLE(127 - 33) AS tt;
SELECT @special_characters;
@special_characters
!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~
Expand Down
28 changes: 28 additions & 0 deletions mysql-test/suite/percona/include/percona_sequence_table.inc
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# $st_field_expression - the list of fileds which will be passed to a SELECT $st_field_expression FROM ...
# $st_table_expression - a table function expression which will be passed to a SELECT ... FROM $st_table_expression
# (do not use single quotes here as they will not be interpolated properly by assert.inc)
# $st_where - optional WHERE clause that will be added to SELECT $st_field_expression FROM $st_table_expression
# $st_expected_number_of_records - an expected number of records returned
# $st_expected_signature - a comma-separated list of expected returned values
# $st_sort_order - sort_order: <empty>, ASC or DESC (used in GROUP_CONCAT() ORDER BY clause)

--let $st_query_suffix = FROM $st_table_expression
if ($st_where)
{
--let $st_query_suffix = $st_query_suffix WHERE $st_where
}

--echo ### SELECT $st_field_expression $st_query_suffix ###
--eval SELECT $st_field_expression $st_query_suffix
--let $assert_cond = "[SELECT COUNT(*) $st_query_suffix]" = $st_expected_number_of_records
--let $assert_text = [SELECT COUNT(*) $st_query_suffix] should return $st_expected_number_of_records records
--source include/assert.inc

--let $assert_cond = "[SELECT IFNULL(GROUP_CONCAT($st_field_expression ORDER BY 1 $st_sort_order SEPARATOR ","), "") $st_query_suffix]" = "$st_expected_signature"
--let $assert_text = [SELECT $st_field_expression $st_query_suffix] should return "$st_expected_signature"
--source include/assert.inc

--eval EXPLAIN SELECT $st_field_expression $st_query_suffix
--eval EXPLAIN FORMAT=TREE SELECT $st_field_expression $st_query_suffix

--echo
Loading

0 comments on commit e23e6c3

Please sign in to comment.