This is a periodic rollup release. It contains many features, bug fixes, and improvements.

The core API has still not stabilized and will not be guaranteed until 1.0.

Merged PRs

go-mysql-server

1056: Allow REGEXP args to be converted
fix for #1055
1054: Fix prep AS OF
Prepared AS OFs errored when the asof target was not a bindvar. All of our previous tests treated ASOF also as a bindvar.
companion PR: dolthub/dolt#3592
1053: Added type wire tests
This enforces that we're returning the correct data over the wire (which may differ from the storage values). Majority of the fixes made have been moved to the parent PR (#1044) so that this one is not quite as large. They're all fairly straightforward, hence I don't think they require review.
The expected test output has been validated against MySQL (using the shim). In addition, this also enforces that the results are coming from the Type.SQL() function by running the output of a sql.RowIter (gathered from the directly querying the engine) through the Type.SQL() function and ensuring it matches.
1052: enginetest: Parameterize query plan tests
1051: Better join commutativity count
We excluded joins on the basis of join factors in the logical join tree,
rather than distinct join subtrees subject to commutativity. The
difference is that we have to permute n! to optimize the search space
of all valid commutative trees, versus a k^n (k = table rows) execution
runtime. 12! blocks analysis, k^12 is steep but can be OK depending on
cardinality and indexes. We impose no limits on k^n joins.
1047: sql/parse: Test round-tripping sql.Type as string
1046: Introduced OrderedIndex interface to deal with indexes that don't return ordered results
1044: Type value changes & row type assertions
This PR has two primary goals:
1. All types now pass their values around in a form most suitable to that type.
2. Enforce that the aforementioned types are always passed to integrators, such that integrators do not need to do type validation on their end.
  To elaborate on these points, some types already passed around values that were sensible and a best fit for that type, such as MEDIUMINT returning an int32. Other types, such as DECIMAL, passed around strings, which necessitated conversions for integrators to be able to properly persist the values. Not only that, there is currently no guarantee that a row's values each have their best fit type (a BIGINT can work with an int8, but it should be able to always expect int64). To make these guarantees, I'm adding a check at all GMS-integrator junctions that pass a sql.Row and verifying that the value types are exactly what that column expects.
  This may have the side effect of changing the output of a SELECT statement for integrators. As a SELECT statement simply returns a sql.RowIter, additional logic will be needed to convert all values to their canonical MySQL representation. This can easily be achieved by passing all values through Type.SQL() before display.
1043: enginetest: Validating Enginetest Harness
1042: exclude procedure aliases from showing information_schema.routines table
Not show procedure aliases in information_schema.routines table
Dolt PR tests this change
1041: cleanup enginetest
Splits up enginetest/enginetest.go into initialization.go and evaluation.go. The intent is to leave only test banks in enginetest.go
This is a pure refactor, no logical changes were made.
1038: Added a few foreign key tests
1037: Parallelize IndexedTableAccess on DoltDiff tables
1033: fix drop current database with case insensitive name
1032: Bug fixes for ModifyColumn related to primary keys
1029: fix some column values of information_schema.columns table
Fixes column_key, character_maximum_length, data_type and column_type column values of information_schema.columns table
1027: Moved responsibility for rewriting tables on add / drop primary key into engine
1026: More tests converted to new format
1025: adds mod() function
fix for: dolthub/dolt#3423
1022: Rewrite table for drop column
1021: Allow SHOW CREATE PROCEDURE for external procedures + display a fake CREATE PROCEDURE
A note regarding the fake CREATE PROCEDURE. It seems that Java's JDBC checks that the procedure creation statement is a valid statement, regardless of whether the statement remotely matches that of a stored procedure. So rather than returning a random statement, we return a "valid" CREATE PROCEDURE statement.
Fixes dolthub/dolt#3428 and dolthub/dolt#3424
1020: example memory.NewTable add "PrimaryKey:true" flag.
example memory.NewTable add "PrimaryKey:true" flag.
1019: UUID function is no longer based off current time
fix for: dolthub/dolt#3323
1018: support SRID for spatial type column definition
Added SRID value syntax functionality for column definition.
Added tests cover CREATE TABLE, ALTER TABLE ADD/MODIFY COLUMN, and INSERT statements.
Fixes dolthub/dolt#3425
1015: Thread safe procedure cache
1014: Added field for connected db to logger
1013: Moving transaction initialization before query analysis
Moves transaction creation ahead of query analysis, so that queries can be executed with the latest committed state from other transactions.
Fixes: dolthub/dolt#3402
1011: Add FK Checks for Insert on Duplicate
1009: sql/analyzer: Fix reresolveTables to not drop IndexedTableAccess and DeferredAsOfTable nodes on its transform.
enginetest: Add a test to assert IndexedTableAccess in a prepared statement
behaves correctly when querying a table that has been modified since the
prepare.
1008: Bug fix for rewriting table during modify column
1007: Refactor grant_tables to mysql_db
Mostly renaming variables and files to better reflect what they are now.
Added flatbuffer files for MySQL DB.
1005: Allowing "DEFAULT NULL" for blob column definitions
Fixes: dolthub/dolt#3388
1004: Support for rewriting tables on column modifications
1003: Replace statements that order by primary key, to use indexed table access
1002: New interface to rewrite table on certain schema change operations
Interface isn't quite final but this can be checked in. Complementary dolt changes are done and tested but want to get feedback on this approach first.
1000: Regression test case for dolthub/dolt#3247
dolthub/dolt#3247 reported a panic that our test cases didn't cover. Another commit (324e43b) already fixed the panic, so this PR just adds a quick test to ensure we don't regress with the same bug.
997: adding COLLATE to SHOW CREATE TABLE
Fix for: dolthub/dolt#3351
Also fix for: dolthub/dolt#3352
995: CheckpointHarness and analyzer mutexes
992: Allow unresolved tables and procedures in trigger body in CREATE TRIGGER
In order to allow non-existent tables and non-exitent procedures in trigger body in CREATE TRIGGER statement, we no longer run analyzer on trigger body, instead all validation checks are performed on a single rule, validateCreateTrigger.
Added tests for cases of different action times and events.
990: support SELECT INTO 'variable' functionality
Added functionality support for SELECT INTO variable only. Usages of 'outfile' or 'dumpfile' will return unsupported syntax error.
Added tests for different cases of SELECT INTO syntax. Using SELECT INTO around unions will not work unless the result is a single row because we do not support order by and limit clauses for union statements even though we parse it correctly.
988: Fix SHOW CREATE ... AS OF ... prepared statements
Fix for: dolthub/dolt#3304
Note:
- The parseColumnDefaults rule is repeated because ColumnDefaults are unresolved after unresolveTables and setTargetSchemas, for some reason this only happens when using a DoltHarness.
986: support XOR logic operator
Added XOR MySQL logic operator
985: Fix Handler queries when Prepared Statements are Disabled
Fix for: dolthub/dolt#3305
983: Change Autocommit Query Plan
This PR adds a new analyzer rule and query Node for autocommit. This fixes an issue where previously a query process was being killed (via a cancelCtx function) before a transaction committed.
982: adding flag options 2 and 4 for st_asgeojson
fix for: dolthub/dolt#3280
981: fix panics on CREATE PROCEDURE queries
- Skips evaluating procedure-parameters in the analyzer step
- Added case for handling UnresolvedFunction in fds to avoid panics
- Fixes issue, dolthub/dolt#3241 (when procedure parameters are used in filter expressions, it needs to be evaluated, but procedure parameters cannot be evaluated without nil reference map)
980: Add tests for handling join in parentheses
Tests regarding this issue, dolthub/dolt#2203
Some cases give incorrect result in Dolt.
979: Fixed overly aggressive type change check with foreign keys
Previously PR #977
The bug has existed since the GMS foreign key implementation, but only surfaced after the previous PR. When changing a column's type, I was only checking that the table was used in foreign keys, when I needed to check that the column is used. This fixes that bug.
978: Use transactional logic to rollback bad triggers, also tests
Added a new rule that wraps everything in a TriggerRollback node if there is a trigger present. This new node will make a savepoint right before execution, and will return to normal if something breaks.
977: Foreign keys now allow limited type changing
It seems that, specifically when using the string types, you may lengthen a field, but you may not change any other properties (cannot change character set or collation). Non-string types don't seem to allow lengthening, as you can't change an INT to a BIGINT.
The implemented behavior is actually an approximation of MySQL's, as you cannot cross some length thresholds due to how MySQL stores strings. From what I can gather for MySQL, as long as the on-disk data does not change AND does not need to be checked, it is allowed. I'm sure more type changes are allowed, but it's not documented, and therefore would require a ton of trial and error, so just shipping this for now.
975: Fixing return types for spatial functions
Fix for: dolthub/dolt#3279
Adding tests that check the return type for functions
- st_asgeojson
- st_aswkb
- st_aswkt
- st_astext
- st_dimension
- st_latitude
- st_longitude
- st_swapxy
- st_x
- st_y
974: Fixed LOAD DATA errors
There are three core issues with LOAD DATA that have been fixed:
1. Columns could be rearranged, but we didn't handle the case where the rearranged columns were a subset.
2. Line endings were assumed to be a single character, which is not true for Windows (which uses \r\n).
3. Default values did not work. Nil default values "worked" due to nil being the default value when instantiating a slice of interface{}s.
971: support handling join table between parentheses
Allow handling join tables in parentheses. It causes the JoinTableExpr to be wrapped in ParenTableExpr in the parser, so it needs to be unwrapped to get the JoinTableExpr inside
Fixes dolthub/dolt#3254
970: Changed all DDL nodes to return an OkResult
This is important for compatibility with certain SQL clients (notably C connector).
969: Improving Error Message for Inserting Strings that are too Long
includes the string that failed and for which column
helps for: dolthub/dolt#3270
968: Expanded AddColumn tests to include some writes on the new table
967: add TRADITIONAL to sql_mode set
Added TRADITIONAL option to sql_mode set.
Fixes dolthub/dolt#3230
966: Arithmetic .Type() Perf Improvement
SQL Logictests were timing out on huge CASE statements with ~40
nodes. This fixes the computational blowup, which I think was
somehow exponential before.
965: Bug Fix: FilterTable pushdown to aliased tables
Fixes: #808
Max – Tagging you for your analyzer expertise. Could you check this out and see if you see a better approach to solving this? I saw a couple ways to address this, but this seemed like the least intrusive approach.
964: support more than 1 byte characters to insert for char()/varchar() types
Allows inserting strings with 'exact number of character but consists of more bytes' values to char() or varchar() types
Fixes dolthub/dolt#3233
961: Fix underspecified insert tests
No inserts were not being run with prepareds. A lot of them were broken, but should be running in GMS now at least.
Edit: These pass Dolt also.
960: Parse charset introducer for string literals for expressions
Fixes parsing charset introducer in front of string literals for handling some expressions such as column default expressions.
Allows only utf8mb4, utf8mb3 and utf8 charsets. If other charsets are used, gives unsupported charset error.
958: Added variable to hide external stored procedures
957: Allow conversion of strings and hex to geometry types
fix for: dolthub/dolt#3231
we should be able to read mysql dumps produced using the --hex-blob flag.
956: Fixing Point method to actually print string
Fix for: dolthub/dolt#3228
Allows the use of point(), linestring(), and polygon() for default values for columns of type point, linestring, polygon, and geometry.
955: allow NULL value for geometry types
Allow handling null values some geometry data types. For these types, DEFAULT NULL values should be allowed
954: Allow arbitrary parameter count for external stored procedures
If an overloaded function contains a variadic variant, that variant is only used if an exact match is not found.
953: fixing geometry tests
Fix for: dolthub/dolt#3217
Types returned should be wrapped in geometry.
952: Remove checks for unique index
We should follow up with an implementation for a unique index for the memory table. Will take a crack at this in a future edit.
951: Added support for external stored procedures
This adds support for hooking functions into the stored procedure system.
There were two considerations for the implementation, and I felt this implementation is superior (using the reflect package). The other primary consideration was to expect a specific function signature that takes a map[string]interface{} representing each parameter, and there'd be another map[string]sql.Type returned in the loading function that specifies the type of each parameter. I didn't like that the types were interfaces, and their definitions declared elsewhere, as I'd rather the engine handle as much as possible. I'd prefer to have this all statically-enforced, but Go's type system is sadly not flexible enough, so this is the next best thing.
The reflection logic is fairly straightforward, and there are many comments, so I'm not too concerned with this being hard to maintain at all.
948: Allow insertion of negative numbers into auto_increment columns
Fix for: dolthub/dolt#3192
Added an additional check to just not run auto_increment logic when value provided is less than the Zero() value for column.
Small change to memory/table.go to stop incrementing the current autoIncVal when the provided value is strictly less than the current autoIncVal.
947: Update drop table tests to add primary keys, Add more ordering to JSON script test results
946: enginetests: Added result ordering to a few more JSON script tests
943: Return correctly formatted column defaults in the information_schema.columns table
The information_schema.columns contains the column column_default that represents the formatted string version of the column default. This correct representation is essential as tools (like table editors) use this column to generate representation of a table or their alter statements.
This pr pushes the resolution of column defaults through the analyzer. Note that there are some open flaws
1. The double quoting of literal column defaults
2. The double parentheses for expressions due to improper parentheses handling in the codebase
3. Missing backticks for columns. This is due to fact that all the way down to GetField for expressions we don't actually backtick the name. We should followup with a rewrite of the Column Default String method
  There should be subsequent followup work that simplifies string formatting with default values and expressions as a whole.
942: enginetest: Update JSON Scripts tests to add primary keys and ordered results
941: Fixes unhashable filter bug, adds semantic validation starter
Fixes unhashable filter bug.
Adds improvement on semantic operand validation. Targetted checks easier
to implement.
940: Enable RowIter2 on IndexedTableAccess
This is pretty limited and only works for static lookups, not joins, but is enough to benchmark with
This change also does some refactoring of index analysis code, but no other functional changes
Small bug fix to the String() method of SubstringIndex
939: ValidateJoinDepth rule; hard cutoff for > 12 table joins
New analyzer rule validateJoinDepth that errors if we count more than 12 join leaves. Opaque node counting could be done more carefully.
join search schema comparison was O(n^4) if i counted correctly, now should be more like O(n)
938: Quickperm, reduce join search memory usage
936: Allow auto_increment on non-primary key columns
Fix for: dolthub/dolt#2981
Since Unique Keys are not a part of sql.Schema and sql.Column, we need to check if there are any valid indexes defined over the columns we want to mark as auto_increment.
Added skipped tests for queries that don't work.
TODO:
- Adding auto_increment adds NOT NULL constraint too
- Support adding columns; need to be able to alter table t add column j int unique first
- Autoincrement on keys defined over multiple columns; MySQL only allows leftmost column to be auto_incremented
935: update create view parsing and added definer var for create trigger stmt
Updated CREATE VIEW parsing supporting algorithm, definer, security syntaxes, but does not support the functionality or affect create view query statement.
Added Definer variable to Trigger.
Added and updated engine tests covering these changes
934: More transform and join tests
932: Adding support for show create table <table> as of <version>
Adds support for show create table <table> as of <version> to go-mysql-server
Depends on:
- dolthub/vitess#151
930: Editing test data to work around failures with new Dolt storage format
Without changing the test data, the new Dolt storage format doesn't recognize that there is any change and it errors out with "nothing to commit" during testdata setup.
Editing the test data allows the testdata setup to complete successfully on the old Dolt storage format and on the new Dolt storage format. TestVersionedQueries is now configured to be skipped on the new Dolt storage format.
929: have _example/main.go work as expected
Fixes: #911
928: Updating versioned query tests based on behavior differences in Dolt harness
When running the versioned query tests with Dolt harness there are two behavior differences that cause these tests to fail. To fix those, we need to:
1. explicitly delete old table data and
2. temporarily remove non-existent version test since Dolt reports a no branch error, while Memory reports a no table error.
927: Prevent Creation of Unique Keys
Small fix for preventing the creation of FULLTEXT keys as well.
TODO: There should also be a change in Dolt to prevent altering and modifying columns to use FULLTEXT keys
926: Have VIEWS show up in information_schema.columns
Fix for: dolthub/dolt#3109
VIEWS now show up, but the our current implementation of ViewDefinition makes it difficult to match MySQL capabilities.
924: InsertInto Bindvars
InsertInto is special in that [Source] is not a child sql.Node,
so [Source] needs to be traversed separately to touch every expression
in a DFS search.
This fixes a bug in that DFS search missing the non-Source node
children.
923: Adding support for an as of expression in show columns from table
Passing through as of expression in show columns statements.
Depends on Vitess support in: dolthub/vitess#150
920: handle more time formats
fix for dolthub/dolt#3102
918: Multi Alter Add/Drop Columns
This pr does the following
1. Modifies Add/Drop/Rename/Modify columns to pull the relevant table fresh from the database
2. Adds error checks in validate_create_table to make sure that invalid columns are detected at analyze time
  This pr does not:
3. Address multi-alter ddls and edge cases with default values. I found a ton of small errors while working through edge cases here that I think there is a good amount of work separate from the immediate multi-alter ddl needs. This pr is enough to get us unblocked on panics that used occur when mass add/drop pairs were created.
915: fix create/drop TRIGGER parsing to support database name specific query
914: Adding Geometry type
Adds geometry type for GMS side.
Includes case for geometry type for functions:
- st_swaxy
- st_srid
- st_dimension
- st_aswkt
- st_aswkb
913: Apply IndexJoins with CrossJoin children
910: Ensure that the local index map uses the computed id
We should use the computed id of index as key for the index map of the memory table. Previously we were using the given index name which would conflict when multiple indexes were assigned the empty string.
909: fix create/drop procedure parsing to support database name specific query
907: Add db name to the column name for show tables statement
- Database name is now included in column name in SHOW TABLES statement
- Fixed cardinality column of information_schema.statistics table returned wrong type before
906: Store the creation time for trigger statements
Will have changes to Dolt side to store this for Dolt Databases.
MySQL has nanosecond precision, which we wont have here.
I mimicked stored procedure behavior creation time for mysqlshim: https://github.com/dolthub/go-mysql-server/blob/main/enginetest/mysqlshim/database.go#L169
904: IndexedJoins mark handled filters
Applying an indexed join in applyJoinPlan should apply equality
lookup expressions, but currently do not. pushdown mistakenly
marked those expressions as handled and eliminated necessary filters.
This PR prevents IndexedJoinAccess with nil lookups from marking
filters as handled.
A future improvement could the pushdown range logic
to opportunistically apply lookups in applyJoinPlan, and let
pushdown mark those as handled. This would improve perf by both
removing both a table scan and filter from some index joins.
903: support show create procedure
899: run query on info schema routines table for SHOW PROCEDURE STATUS
Also fixes SHOW PROCEDURE STATUS to show all procedures in all databases.
Removes show_procedure_status.go file (removing the showProcedureStatus node)
898: Improve MultiAlters error handling with Alter Primary Key, Alter Index, Alter Default, Alter AutoIncrement
This pr does the following
1. Improves MultiAlter DDL analyzer by checking for failure modes in adding/dropping pks, adding/dropping indexes, adding/dropping defaults
2. Pushes the table resolution of AlterPk, AlterIndex, AlterDefault,Alter AutoIncrement at execution time. This is due to the fact that the node should know whether columns were added or dropped in previous steps of a MultiAlter DDL.
3. Adds a small bug fix with how columns were being dropped in the validate_create_table
4. This pr does not include changes to multialters with ADD/DROP constraints due to additional complexity.
897: Code sample update to remove missing Auth package
Updated code sample in README and _example/main.go to remove the missing Auth package.
Reported in: #888
896: adding 1 second to trigger creation time
Fix for this issue: dolthub/dolt#3047
Triggers weren't showing up in TablePlus because the creation time for triggers is always 0 in unix time, which is out of TablePlus supported timestamp range (for some reason). So this PR just makes the trigger creation time 1 second after the start of all time, which TablePlus considers valid.
895: fix SHOW INDEXES FROM otherdb.tab fails as database is initialized incorrectly in parsing
Before Show Index statement was initializing the Database incorrectly as it would set it to either empty or current database. This was an issue for using the statement for database that user is not currently on.
This PR also fixes couple of info schema table inconsistencies against Mysql.
893: /go.mod: require go 1.18
890: support SHOW FUNCTION STATUS functionality
SHOW FUNCTION STATUS result is queried from INFORMATION_SCHEMA.ROUTINES table
887: fix non_unique column value definition in information_schema.statistics table
fix non_unique being assigned incorrect value. Verified with Mysql docs, "0 if the index cannot contain duplicates (which .IsUnique() is true), 1 if it can."
886: Auto Increment Refactor
885: Ensure CheckPrivileges runs on resolved table functions
While testing privilege checks for the new dolt_diff table function, I realized that CheckPrivileges was being run on UnresolvedTableFunction and not the actual resolved table function. Since the dolt_diff table function pulls data from underlying tables, it needs to be resolved before CheckPrivileges runs to ensure we can accurately check privileges.
884: added warnings for some syntaxes of ALTER TABLE and CREATE DATABASE statements
- Added a warning for using CHARACTER SET, COLLATE or ENCRYPTION syntaxes when creating database. The database is still created.
- Added a warning for using DISABLE KEYS or ENABLE KEYS options for alter table statement. Nothing changes for the table being altered.
- Added a system variable that mysqldump uses, innodb_stats_auto_recalc
881: More tests for alter table bugs
1. Split PkOrdinals tests into DDL and DML (new)
2. New script test that triggers the primary key / secondary key ordinal mismatch
3. Add ctx.Done() checks to a few key node iterators before operating on child rows (may have missed some, but tried to hit the table, index, and edit iterators)
  companion PR: dolthub/dolt#3013
878: Fix logic with OR operation and add missing nil check to Like
MySQL has some really weird behavior where the OR of two non bools evaluates too false. Improperly handling this situation was trickling up to a LIKE panic on the new query added to queries.go
```
SELECT "A" OR "A";
+------------+
| "A" OR "A" |
+------------+
|          0 |
+------------+
```
873: Add support for making fulltext return unsupported feature
This pr addresses this issue
Cc mysql docs
872: sql: expose convert native go value to value
Expose convertToValue function from memory package as ConvertToValue.
Return an error instead of panicing if the value type is not known.
871: Apply bindvars to subqueries
Cherry pick from #795 to fix Subquery bindvars specficially.
869: Fixed invalid hex number type association from number to blob
Fixes dolthub/dolt#2968

867: TransformUp is now sensitive to tree modifications
TransformUp and related node/expression DFS helper functions expect the
visit functions to return an additional boolean parameter indicating
whether the visit changed the node:

type TransformNodeFunc func(Node) (Node, bool, error)
type TransformExprFunc func(Expression) (Expression, bool, error)
type Transformer func(TransformContext) (sql.Node, bool, error)

TransformUp's implementation uses the modification information to avoid
re-creating a node with identical children:

BenchmarkTransformOld
BenchmarkTransformOld-12          	  396544	      2782 ns/op	    3000 B/op	      51 allocs/op
BenchmarkTransformOldNoEdit
BenchmarkTransformOldNoEdit-12    	  407797	      2731 ns/op	    2936 B/op	      50 allocs/op
BenchmarkTransformNew
BenchmarkTransformNew-12          	 4584258	       254.1 ns/op	      96 B/op	       5 allocs/op
BenchmarkTransformNewNoEdit
BenchmarkTransformNewNoEdit-12       	 4782098	       237.8 ns/op	      96 B/op	       5 allocs/op

We use plan.InspectUp when possible, and then plan.TransformUp
where possible, resorting to the more expensive plan.TransformUpCtx
and plan.TransformUpCtxSchema only when necessary.

862: fix misspelled name in information_schema.innodb table
861: have methods for dolt to call to parse check expression
To properly merge check constraints, Dolt needs to partially parse their expression strings.
This PR gives Dolt access to some helper methods, to complete this operation. Should have no functional effect on GMS.
860: Add all non INNODB information_schema tables.
This PR adds a bunch of information_schema tables that were previously missing. All are initialized with an empty row iter.
859: add all information_schema.innodb_* tables as empty tables
858: New Foreign Key Implementation
Baseline PR containing foreign key progress. As foreign keys won't be ready to merge until they're fully completed, this PR will serve as the baseline that future foreign key PRs are based on, so that changes are digestible. Once a PR is approved, it will be added to this one. In addition, this serves as a synchronization point for merging main changes to manage conflicts.
856: Only skip updating defaults when added column has no default value AND is nullable
Fix for bad PR #855 which fails the following Dolt test:
http://github.com/dolthub/dolt/blob/51987ab84ef9d07b3c6ceae3ab65b9ca65ab55bd/go/libraries/doltcore/sqle/sqlddl_test.go#L482-L482
Verified this PR passes Dolt tests.
Also, removes TestAddAndDropColumn test which will be replaced with an appropriate bat test on Dolt.
Related to dolthub/dolt#2939
855: Don't apply defaults to all rows when the added column has no default
Related to dolthub/dolt#2939
When a user was working with a large table, adding a column with no default caused an OOM crash. The crash is a result of the ADD COLUMN sql statement causing every row to be modified in the table. This is not-expected and a bug. We expect adding a column with no default to only modify the schema and not any Dolt row.
The fix is to skip applying defaults to table if the added column has no default. We can call this a "fast" add column since only the schema has to be modified.
In the future, we may also add a "fast" drop column where only the schema is modified. Currently, we modify every row for a DROP COLUMN statement. In the "fast" drop scenario, it might be important to ensure values aren't retained across ADD COLUMN and DROP COLUMN statements with the same column name. This PR also adds tests to ensure this.
854: Pk ordinals delete test
853: removing routine_definition column from information_schema.routines table in enginetest as it gives different values in GMS and Dolt
The value in routine_definition comes out different:
- in Dolt, Exchange(parallelism=3)\n └─ Project(6)\n └─ Table(dual)\n
- in GMS, Project(6)\n └─ Table(dual)\n
852: add information_schema.routines table
851: add collation_character_set_applicability table in infoSchema
850: Added sql.FilteredIndex to allow integrators to specify index filter pushdowns
849: Implemented sort node, number type, and handler support for Row2, Expr2, Type2
848: Add FLUSH PRIVILEGES support
846: analyzer: Collect all range expressions for multi-column indexes
845: Fix InTuple optimizer bug
Last week we fixed a dropped optimizer error return that was masking a
tuple bug. We expected a slice of interfaces, but single element tuples
return one value. We catch this in most circumstances, but missed the
multicolumn case triggered by an AND expression. The error was passing
logictests because we still return correct results even if we fail to
push a filter into a table scan.
842: /.github/workflows/bump-dependency.yaml: pin go version
841: Bump vitess while automatic bumping is broken
840: /.github/workflows/bump-dependency.yaml: add GOOS env
839: server: Refactor handler to use buffered channel
838: Table function support
Adds support for table_functions in go-mysql-server.
Depends on:
- dolthub/vitess#131
837: Add IGNORE behavior for CHECK CONSTRAINTS and on duplicate key
Fixes an issue where INSERT IGNORE was not correctly ignoring check constraints
836: fix set serialization
833: Fixing a race condition by no longer updating the UserVar's type during Eval
The type was being set when retrieved in Eval for completeness, but it shouldn't be needed.
When used as a value, the analyzer was already using the NewUserVarWithType method that explicitly loaded and set the type. The NewUserVar method (which doesn't set type) should only be used when the user var is the left hand side of a set var operation.
832: add processlist table to information_schema
The result is the same as current SHOW PROCESSLIST query result
829: Added additional privilege tests
Added new testing structure for privileges I'm calling "quick tests". A test can take up 6 lines rather than 18, which is a big difference when there's 100 of them. Plus they're quicker to write this way. They're the same is normal tests though.
828: Correct primary key column output for show create table
Used the column order before, instead of the key order
827: fix ROLLBACK; fail with no database selected
MySQL lets ROLLBACK when no database is selected
826: Don't call ProcessList.Update on every row spooled in a result set by default
825: add information_schema.statistics table
Cardinality is currently set to total number of row in the table instead of an estimate number of unique values in the index
824: analyzer doesn't error for invalid value
re: #815
Originally we were failing to handle an analyzer error. Catching that error exposed a case where we want to manually prevent that error path from bubbling up.
823: sql: Added destination buffer parameter to Type.SQL()

820: Add uint types to LiteralToInt
Re: issue #799
This database and this query were parsing 128 as a uint8, which my type switch missed:

$ dolt clone https://doltremoteapi.dolthub.com/post-no-preference/stocks
$ cd stocks
$ dolt sql
stocks> select date, act_symbol, avg(close) OVER (PARTITION BY act_symbol ORDER BY date ROWS BETWEEN 128 PRECEDING AND CURRENT ROW) AS ma200 FROM ohlcv WHERE act_symbol='AAPL' having date = '2022-02-11';
offset must be a non-negative integer; found: 128

After fix:

select date, act_symbol, avg(close) OVER (PARTITION BY act_symbol ORDER BY date ROWS BETWEEN 128 PRECEDING AND CURRENT ROW) AS ma200 FROM ohlcv WHERE act_symbol='AAPL' having date = '2022-02-11';
+-------------------------------+------------+--------------------+
| date                          | act_symbol | ma200              |
+-------------------------------+------------+--------------------+
| 2022-02-11 00:00:00 +0000 UTC | AAPL       | 158.29837209302272 |
+-------------------------------+------------+--------------------+

I haven't been able to create a testing database with the same type parsing behavior yet. Something about the stocks database or that specific query is yielding a types.Value with value=128 and type=sql.Uint8.

819: fix BETWEEN function of missing inclusive logic
BETWEEN() does inclusive check including checks with NULL value
817: Changing UserVar to return its correct type
Changing UserVar to return its correct type, when used as a value in an expression.
Resolves: #790
816: /go.{mod,sum}: bump vitess
813: /sql/analyzer: Added feature flag for single threaded execution
811: fix DropTable enginetest
fix DropTableTest in enginetest, was causing failed test in Dolt
810: Extending script queries to allow easier testing of warning messages
This change lets you specify the expected number of warnings and an expected substring from test queries. It also tidies up some of the logic around running tests with expected warnings.
Example test syntax this allows you to write:
```
Assertions: []enginetest.ScriptTestAssertion{
{
Query:                           "select * from dolt_diff_t;",
ExpectedWarning:                 1105,
ExpectedWarningsCount:           4,
ExpectedWarningMessageSubstring: "unable to coerce value from field",
SkipResultsCheck:			true,
},
},
```
Needed for: dolthub/dolt#2832
809: query plan test for duplicate indexes
806: Deduplicate IndexLookup filters
Opportunistically eliminate filter expressions when
sql.IndexedTableAccess is used for a table scan. The original logic
respecting join scoping for applying indexes is unaffected, this simply
connects out current filter elimination logic with the expressions used
for range scan.
This requires integrators implementing sql.IndexLookup to only return
rows selected by the lookup filter.
Changes were made to ranges and datetime filters To allow integrators
to comply to this filtering standard. GMS can now distinguish between
NULL and infinity. Tests are added for IS NULL and IS NOT NULL handling
for single and multi column expressions. Engine datetimes are not truncated
to support date index lookups with the same precision as MySQL.
We currently fail to eliminate filter inversions. More work is needed
to compare inverted filters to their range scan representation.
804: Revert "addColumn"
This reverts commit f64b718.
803: Add NamedWindows Node
This moves window name validation, resolving, and embedding out of
parsing and into the analyzer.
802: Add vitess to go mod to support SHOW STATUS LIKE
801: Added external functions for persisting Grant Tables
This adds the functions necessary to be able to easily persist the data from Grant Tables as JSON for an integrator.
797: Added privilege checks for every currently supported statement
This adds privileges for every supported statement.
796: Framework for RowIter2 and other new data representation interfaces
- Implement Table2 for memory tables
- Several low-level node types converted to Node2
- Special top-level iterator to switch iteration type based on whether the tree is all Node2 or not
795: ComPrepare saves partial query plan
Prepared statements hidden behind enable_prepared_statements config.
Lacking:
- Full support for versioned questions (select * asof '2021-03-21)). I fixed the panics, but many asof tests return incorrect results.
- Window function parameters can't be bindVars currently (lag(x,?))
- Needs a lot more insert ... select tests
  ComPrepare saves a partially optimized plan for prepared statements.
  Query execution will use the saved plan when possible. Connections have
  a private view of saved plans that are deleted on termination.
  All enginetests ran with prepared queries regarless of BindVar
  inclusion. Variety of bug fixes exposed by the prepared enginetests.
794: Fix drop table from different database
Added DropTable case in getTransactionDatabase
Added tests to check dropping table in out of scope database
Changed ddlNode to CurDatabase that is update with every table being dropped
Changed taking array of tables as string type to plan.UnresolvedTable type
Updated resolve-table rule in analyzer to support DropTable filtering out non-existent tables.
793: Add type checking to auto increment
792: Named Windows
Build WINDOW clause definitions from AST into plan.Window nodes.
Coversion currently happens in parser. This breaks the separation of
concerns between parsing and semantic validation to avoid creating new node
types to differentiate between Over windows and Window clause windows.
A proper refactor probably requires changing the the parse/exec
representations for all related nodes to carry info from the parser
further before semantic validation.
791: Database and table name visibility affected by privileges
This limits the visibility of database and table names depending on the privileges associated with an account (whether directly on via an active role). In addition, all engine tests now run with grant tables enabled. Of course not all statements are checking for their respective privilege, but all of the other logic paths are being exercised (especially the new PrivilegedDatabase and ...Provider constructs).
I removed some driver implementation as it was literally broken (errors in the file on main) and was obviously not relied on by anything (not even tests), so instead of fixing it for the changes I made, I just removed it altogether.
789: fix alter table columns in different database
Removed ddlNode and added tests for AddColumn , DropColumn , RenameColumn , DropColumn that check updates with different database tables than current one.
Added cases in engine.go for updating the transaction database for some plans including above and CreateIndex , AlterIndex
786: Added all static global, database, and table privileges
This adds all static global, database, and table level privileges. They parse and persist but do not yet restrict on anything. Future PR will dive into the full set of privilege checking.
One major point to note (which will probably bring about confusion otherwise) is that these new tables are implemented as essentially a "view" on top of the mysql.user table. This will significantly help when it comes privilege checking, especially in the context of determining active roles and such. To complement this, PrivilegeSet was almost entirely rewritten. The related files are data_editor_view.go and grant_table_shim.go.
With the original direction, I could have 10 different table queries and 10 sets of privilege data to manage with only 2 roles. This way, I'll only have 3 queries and 3 sets of privilege data in the same scenario.
785: No longer mark subqueries in triggers as cacheable
Resolves issue dolthub/dolt#2674
784: fix alter table add pk
Added case for getting non current database name in transaction for AlterPk
783: recursive CTEs
RecursiveCte node impl with tests for joins, group by, and subquery
aliasing.
RecursiveCte's are not supported in DML statements. Nested recursive
CTE's and nested subqueries are not supported yet. Aliasing is not
neatly split down the middle on either side for RCte unions; only the
dual table can have alias clashes on either side of the CTE for now.
Cyclical recursion will timeout at depth 20. This will likely need a
rewrite with better global aliasing to support infinitely nestable
RecursiveCte's.
Indexed table lookups are not applied to RecursiveCte tables currently
because of a bug in the memory impl.
782: migrate invalidArgument in time.go to errors.go
781: Expose Typ field in sql.RangeColumnExpr
780: Expose enginetest.CreateIndex() for testing
779: fix json_contains error handling
Fixed tests for "Fix JSON Extract on struct" PR
778: Properly implemented SHOW GRANTS and SHOW PRIVILEGES
SHOW GRANTS was a shell that always returned the same result, and it has now been properly implemented (without USING since active roles aren't in yet). Also added SHOW PRIVILEGES.
777: Add PeerGrouper, simplify count and rank window funcs.
Count has a default range framer now, matching MySQL and
Postgres's behavior. Move custom peer grouping logic into
a formal framer to simplify ranking agg functions, which
should simplify other rank funcs in the future.
775: add groupby column check
Any alias column that is non alias table column will not have table name assigned.
774: Prevent invalidating table check constraints when altering table schema
Updated logic for dropping and renaming columns to ensure that no table checks are referenced.
Further improvements should eventually be made to ensure that table check constraints aren't modified by other schema alterations, such as modifying a column's type.
773: Added roles, revoking, and dropping users and roles
This adds roles, along with implementing the accompanying statements and functionality. In addition, we may now revoke privileges and roles, and also drop users and roles.
772: ST_LATITUDE and ST_LONGITUDE functions
771: RANGE window framing with rangeFrameBase
Support RANGE window frames, adds unit tests for framing
and enginetests for queries.
Count needs to be rewritten to support RANGES. Better validation
for range expression/bound matching would be helpful.
770: fix not between function and or logics
Fixed handling NOT BETWEEN function was incorrect
Added missing logic for OR operator
769: Fixed a bug (just introduced) in resolving default column expressions
Unbreaks dolt
The reason that this bug didn't break the in-memory implementation is it keeps its column defaults reified as Expressions, so it doesn't really use most of the related analyzer machinery
768: go.mod,.github: Bump to 1.17.
766: Fix panics and errors introduced by recent changes to column default resolution
This will fix dolthub/dolt#2659 when picked up by dolt
764: Window frame support
This PR adds sql.WindowFrame and sql.WindowFramers to represent
window frames from AST -> plan -> exec.
Every unique and valid combination of window frame
Start and End bounds has a plan node (sql.WindowFrame)
and a framer (sql.WindowFramer) pair. The frames, framers,
and constructor functions that map from AST -> plan and
plan -> exec are generated by optgen.
763: Basic privilege granting and checking
This adds an extremely basic implementation of privilege checking, along with the ability to grant a subset of privileges. In addition, connections carry along their authentication information, and direct queries on the engine (through use of the *sql.Context) also carry auth information for later privilege checking.
762: Refactored how grant tables are stored
This refactors a lot of previously-written code to more manageably handle future modifications to the grant tables, and to ease their use and reduce potential development mistakes.
761: ST_SWAPXY function
760: ST_DIMENSION function
TODO: missing cases for multi geometry types
759: Revert "add ability to override logger (#758)"
This reverts commit 2a4f3ea.
758: add ability to override logger
756: Adding support for non-string key types in JSON_OBJECT
Resolves: dolthub/dolt#2626
755: sql/expression/function: json_{array,object}.go: Fix to handle JSONValue subexpressions as well.
753: go.mod: Bump vitess, pick up parser simplifications.
749: GroupBy normalization rule maintains parent Projection dependencies
We flatten GroupBy aggregations to isolation agg functions, but in the process lose nested column dependencies. The downstream qualifyColumns rule was erroring, but without passthrough projections binary expressions like Arithemtic(Sum(x.i), y.i) will fail at execution time without a full set of input dependencies.
For this query:`
```
select sum(x.i) + y.i from mytable as x, mytable as y where x.i = y.i GROUP BY x.i
```
We correctly identify that the GroupBy node has one primary aggregation, Sum, and project the Arithmetic separately.
```
GroupBy -> Sum(x.i) + y.i -> ... TableScan (x,y)
=>
Project (Arithmetic(sum(x.i) + (y.i)))-> GroupBy(Sum(x.i)) -> ...
```
The Project node fails downstream trying to lookup the y.i dependency we discarded in the transform. This PR adds dependencies back to cover the new parent Project for GroupBy flattening.
```
GroupBy -> Sum(x.i) + y.i -> ... TableScan (x,y)
=>
Project (Arithmetic(sum(x.i) + (y.i)))-> GroupBy(Sum(x.i), y.i) -> ...
```
746: Window exec uses framing iterator and support agg functions in windows
plan.Window is converted into new WindowIter backed by the framing setup.
Codegen unary agg function nodes, including executable and unit testing
harness. Rewrite window aggregation functions for framing.
More unit tests that will make it easier to debug WindowBlock/Iter
Doc comments for the new WindowIter exec stack.
745: Added password validation for logging in
Now when creating a user, you can specify a password and log in according to the password given. Denies all incorrect password or invalid users. Also added a killswitch in the grant tables to disable this functionality since it's not yet ready.
744: /enginetest: Add a primary key to 'specialtable'
743: Fixes pushdownGroupByAliases bug with repeated projection column name.
The analyzer rule stored column names in a hash map to mirror deps on
either side of the GroupBy and Project child node. The return expressions
were recreated without the table info, which is fine for most cases but
breaks when there are deps from two tables with the same column name.
This maintains table metadata for *expression.UnresolvedColumn
throughout the rule, so that the output projections maintain table
identity.
741: add JSON_ARRAY() function
740: one more reentrant lock removal
739: /sql/analyzer: Remove reentrant locking from Catalog
737: Adding functions st_asgeojson and st_geomfromgeojson
736: Fixed bug for JSON values in CASE expressions
735: server/context.go: getOrCreateSession: Use a safer .mu.Unlock construct in case s.NewSession panics.
734: Fixed example in README
Fix mysql command line mismatch the example code
733: add JSON_MERGE_PRESERVE() function
732: Allow conversion of zero values of other types to date / timestamp values
This is for compatibility with postgres dumps and other tools that expect it
731: Revert groupby change for partitionless agg functions
GroupBy can't switch it's execution runtime to the same as Window without a significant perf hit for select count(primary_key) from table. We could simplify group by iter/its aggregation functions now that it's just a for loop off a child iter.
729: Lazy evaluation for column defaults
728: Adding SRID and Axis Options for WKT and WKB functions
727: Ensures that check constraints cant be added to table that violate th…
…e rule
726: add int uint conversion
724: GroupBy window framing refactor
This PR adds:
- design for window framing
- new agg functions for window frames
- GroupBy refactor using framing (frame = partition)
  Aggregations will have two execution paths while I am swapping out the Window execution layer:
- In GroupBy node, agg functions will use window frame setup
- In Window nodes, agg functions will continue to use AggregationBuffer
  Other:
- The aggregation package is a bit of a mess right now. I will delete all of the old aggregation and window code during the window refactor.
- I was 50/50 on whether or not we should use a special iterator for groupBy. It was not too hard to extend the windowBlockIterator, which should drop-in work for regular windows.
723: Apply Defaults in the AddColumn node
This pr
1. Adds Defaults in the Add Column Node
2. Adds a nullability check in the Update node
722: Bug fix for using TRIM function in CHECK constraints
721: Adding Support for SRID
Adds srid field to existing spatial types, and st_srid function: https://dev.mysql.com/doc/refman/8.0/en/gis-general-property-functions.html#function_st-srid.
Will add SRID option to wkb and wkt function in separate PR.
720: Allow deterministic functions in CHECK constraints
719: Fixed a bug in trigger execution involving update statements in BEGIN blocks
This fixes dolthub/dolt#2534
718: Adding Well Known Binary Conversion Functions
Functions:
- st_asbinary
- st_aswkb
- st_pointfromwkb
- st_linefromwkb
- st_polyfromwkb
- st_geomfromwkb
717: add TIMESTAMPDIFF() function
716: Detect and give a reasonable error message for SHOW TABLES with no current db
Fixes dolthub/dolt#2559
This also moves the resolve databases rule into a different analyzer phase and fixes a string of issues that arise when that move occurs.
715: Add lag function.
Spec: LAG(expr, offset(default = 1), default(default=nil)).
712: INSERT IGNORE data conversion behavior
This PR addresses behavior for INSERT IGNORE with regards to data conversion errors.
1. Changes column value to 0 values on incorrect types
2. Truncates strings that are too long
  cc. https://dev.mysql.com/doc/refman/8.0/en/sql-mode.html#sql-mode-strict
711: Adding Well Known Text Conversions
These are functions that allow a user to create spatial types using the Well Known Text (WKT) format.
Functions:
- point_from_text
- line_from_text
- poly_from_text
- geom_from_text
- as_text
- as_wkt
710: Adding functions ST_X and ST_Y
These are spatial type functions that work specifically with point.
708: sql/parse: Update sql/parse/parse.go for dropping vitess partial ddl support.
704: Basic CREATE USER implementation
This adds the minimum for the CREATE USER statement to work, in that we have an in-memory representation of the grant tables, which may be accessed either directly (by the mysql database) or through the user statements (of which we only have CREATE USER for now).
701: add DATEDIFF() func
699: server/handler.go: Close the row iter in doQuery with the same context we run the query with.
If a row iter implementation captures the |ctx| from RowIter() or Next() calls,
and that |ctx| is canceled before the Close() call happens, then the
implementation may return an error on the canceled context. It is better for
our existing implementations if the Close() calls happen with the same context
as RowIter() and Next().
697: sql/analyzer: Fix a bug where we do not use indexed joins for some queries with offset or sort+limit[+offset].
696: Fix boolean SQL conversion panic
Fixes a panic happening when returning Boolean values from the database.
695: Add SQL func time_format
Add SQL func time_format
Also took the time to refactor time parsing to match MySQL behavior closer, even its weirdest parts.
Fixes a bug in Time.Convert that led single-digit micro-second times to be parsed as tenth of seconds, e.g. 14:00:00.00007 was parsed as 14:00:00.7.
Note: I'm trying to achieve minimum compatibility with Metabase's MySQL connector, which required some functions (see #694)
694: Add Locate SQL function
Add Locate SQL function.
I'm trying to achieve minimum compatibility with Metabase's MySQL connector, and it requires a couple new functions, so you'll see a few more PRs from me.
Reference: https://www.w3schools.com/sql/func_mysql_locate.asp
693: Parsing statements dealing with users and privileges
This just parses all of the statements from their vitess representation to their GMS representation.
691: sql: RowIter,IndexKeyIter,etc.: Add ctx parameter to Next().
690: Adding spatial types: point, linestring, and polygon
Able to create table with column of point, linestring, and polygon type.
Able to call some geometry constructors to create and display a point, linestring, and polygon.
Added functions: st_pointfromtext, st_linestringfromtext, st_polygonfromtext and tests
689: Added test utility method for IndexTableAccess
686: server/handler,sql/plan/row_update_accumulator: Move the logic for generating proper OkResult values for RowsAffected and InsertId to the engine.
This makes RowUpdateAccumulator responsible for returning the right OkResult,
instead of having the server/handler hack up the values in the result after the
fact. A few enginetests needed to be updated because they asserted the wrong
values for InsertId coming back in the OkResult.
It also makes RowUpdateAccumulator responsible for setting FoundRows, RowCount
and LastInsertId. LastInsertId handling is pretty gnarly becuase we currently
don't have it coming back from the insert iterator. It should be moved there.
685: server event listener
684: sql/plan: load_data: Move LoadInfile logic from server/handler to the plan node.
This puts the LoadInfile functionality as a Service on the sql.Context and has
the plan node call into it. This is a better place for it in general, rather
than inspecting the resulting plan tree in server/handler and calling adhoc
methods on the *mysql.Conn based on what we find.
Also addresses a number of (potential) issues with the existing load data local
functionality including:
- Race-y file writes and reads from a single file (.LOADDATA) for concurrent
  requests.
- Incorrect handling of scanner.Err() checking; we now correctly return the
  errors when they occur, instead of deferring them to the Close() call.
- Avoids sending LoadInfilePacket for a non-LOCAL LOAD DATA call.
  The most complected part of the new implementation is the ordering constraint
  around RowUpdateAccumulator iterator calling Close() on the row source before
  it returns an Ok result from the Next() call. An alternative approach would be
  to copy the contents to a temporary file in LoadData.RowIter().
683: Changed Range overlap algorithm
Implemented a tree structure for ranges based on https://en.wikipedia.org/wiki/Interval_tree#Augmented_tree. I tested this against a benchmark that replicated the slowdown seen in Dolt release 0.34.5 and saw linear scaling, just as we'd hope. There are some cases that are handled very inefficiently, and I left some strategies on how to solve them listed in the comments, which I will probably implement at a later time.
682: Alter table can now modify the auto increment attribute of a column
As part of this work, moved (most of) the validation work out of the query nodes and into the analyzer. Added a bunch of new tests. Fixed some bugs in the memory table this uncovered.
681: Return Uint64 instead of int64 in charsets table
680: Window aggregation buffer sharing test
Re: dolthub/dolt#2448
Dolt diff table aggregation queries were bleeding results. We still use buffer sharing, but this test should prevent regressions for the same issue.
679: server/handler.go: Migrate doQuery rows loop to errgroup.
Previously this used a |quit| signaling channel and an |errChan| to
communicate any errors. We didn't get context cancelation in all cases and our
call to |rows.Close()| raced with the background thread's call to
|rows.Next()|.
678: server,sql/p{arse,lan}: Move handling of KILL statements to plan nodes instead of adhoc regex parsing in server handler.
676: Bug fix for multi-column index matching and test for same
675: Design for RowFrame

673: fix cross join panic
The cross join expression check would panic for any Comparator outside of the whitelist, instead of returning false to indicate an unmatchable expression.

> SELECT a6, x5, b2+a6, e4  FROM t6, t4, t5, t2 WHERE b4 in (434,267,535,407,331,2,921,469)   AND c5=819   AND d6 in (192,129,970,151)   AND (c2=728 OR 806=c2 OR 141=c2)
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x1e2c6d3]
goroutine 1 [running]:
github.com/dolthub/go-mysql-server/sql/expression.(*GetField).Name(...)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/expression/get_field.go:82
github.com/dolthub/go-mysql-server/sql/analyzer.comparisonSatisfiesJoinCondition({0x2d03fbe8, 0xc0013df5e0}, 0xc0013df580)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/analyzer/replace_cross_joins.go:50 +0x2d3
github.com/dolthub/go-mysql-server/sql/analyzer.expressionCoversJoin.func1({0x29e5070, 0xc0013df5e0})
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/analyzer/replace_cross_joins.go:61 +0x5b
github.com/dolthub/go-mysql-server/sql/expression.InspectUp.func1({0x29e5070, 0xc0013df5e0})
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/expression/transform.go:61 +0x39
github.com/dolthub/go-mysql-server/sql/expression.InspectUp.func2({0x29e5070, 0xc0013df5e0})
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/expression/transform.go:68 +0x27
github.com/dolthub/go-mysql-server/sql/expression.TransformUp({0x29e5070, 0xc0013defc0}, 0xc00016dfc8)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/expression/transform.go:53 +0x13d
github.com/dolthub/go-mysql-server/sql/expression.InspectUp({0x29e5070, 0xc0013defc0}, 0xc00016e020)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/expression/transform.go:67 +0xad
github.com/dolthub/go-mysql-server/sql/analyzer.expressionCoversJoin({0x29e5070, 0xc0013defc0}, 0x3cd6108)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/analyzer/replace_cross_joins.go:58 +0x3f
github.com/dolthub/go-mysql-server/sql/analyzer.replaceCrossJoins.func1.1({0x29d3b00, 0xc0013df580})
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/analyzer/replace_cross_joins.go:93 +0x318
github.com/dolthub/go-mysql-server/sql/plan.TransformUp.func1({{0x29d3b00, 0xc0013df580}, {0x29d3b00, 0xc0013df440}, 0x0, {0x37ff6f8, 0x0, 0x0}})
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/plan/transform.go:98 +0x4f
github.com/dolthub/go-mysql-server/sql/plan.transformUpCtx({{0x29d3b00, 0xc0013df420}, {0x29d3b00, 0xc0013df440}, 0x0, {0x37ff6f8, 0x0, 0x0}}, 0x0, 0xc00016e778)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/plan/transform.go:91 +0x6ba
github.com/dolthub/go-mysql-server/sql/plan.transformUpCtx({{0x29d3b00, 0xc0013df440}, {0x29d3b00, 0xc0013df460}, 0x0, {0x37ff6f8, 0x0, 0x0}}, 0x0, 0xc00016e778)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/plan/transform.go:72 +0x41f
github.com/dolthub/go-mysql-server/sql/plan.transformUpCtx({{0x29d3b00, 0xc0013df460}, {0x29d3fc8, 0xc0013df480}, 0x0, {0x37ff6f8, 0x0, 0x0}}, 0x0, 0xc00016e778)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/plan/transform.go:72 +0x41f
github.com/dolthub/go-mysql-server/sql/plan.transformUpCtx({{0x29d3fc8, 0xc0013df480}, {0x0, 0x0}, 0xffffffffffffffff, {0x37ff6f8, 0x0, 0x0}}, 0x0, 0xc00016e778)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/plan/transform.go:72 +0x41f
github.com/dolthub/go-mysql-server/sql/plan.TransformUpCtx({0x29d3fc8, 0xc0013df480}, 0x100c00016e7e8, 0x100e674)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/plan/transform.go:53 +0xbd
github.com/dolthub/go-mysql-server/sql/plan.TransformUp({0x29d3fc8, 0xc0013df480}, 0x203000)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/plan/transform.go:97 +0x45
github.com/dolthub/go-mysql-server/sql/analyzer.replaceCrossJoins.func1({0x29d3fc8, 0xc0013df480})
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/analyzer/replace_cross_joins.go:85 +0x159
github.com/dolthub/go-mysql-server/sql/plan.TransformUp.func1({{0x29d3fc8, 0xc0013df480}, {0x29d45f8, 0xc0013cd200}, 0x0, {0x37ff6f8, 0x0, 0x0}})
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/plan/transform.go:98 +0x4f
github.com/dolthub/go-mysql-server/sql/plan.transformUpCtx({{0x29d3fc8, 0xc0013df2e0}, {0x29d45f8, 0xc0013cd200}, 0x0, {0x37ff6f8, 0x0, 0x0}}, 0x0, 0xc00016eca0)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/plan/transform.go:91 +0x6ba
github.com/dolthub/go-mysql-server/sql/plan.transformUpCtx({{0x29d45f8, 0xc0013cd200}, {0x0, 0x0}, 0xffffffffffffffff, {0x37ff6f8, 0x0, 0x0}}, 0x0, 0xc00016eca0)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/plan/transform.go:72 +0x41f
github.com/dolthub/go-mysql-server/sql/plan.TransformUpCtx({0x29d45f8, 0xc0013cd200}, 0x1d9c087, 0xc0013b6ec0)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/plan/transform.go:53 +0xbd
github.com/dolthub/go-mysql-server/sql/plan.TransformUp({0x29d45f8, 0xc0013cd200}, 0xc0007dbb01)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/plan/transform.go:97 +0x45
github.com/dolthub/go-mysql-server/sql/analyzer.replaceCrossJoins(0x26a3cc2, 0x26a0458, {0x29d45f8, 0xc0013cd200}, 0xc0013cd200)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/analyzer/replace_cross_joins.go:78 +0x45
github.com/dolthub/go-mysql-server/sql/analyzer.(*Batch).evalOnce(0x29d45f8, 0xc00096d740, 0xc0007dbb60, {0x29d45f8, 0xc0013cc5d0}, 0x104e534)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/analyzer/batch.go:93 +0x1e7
github.com/dolthub/go-mysql-server/sql/analyzer.(*Batch).Eval(0xc0006b9350, 0x37ff6f8, 0xc0007dbb60, {0x29d45f8, 0xc0013cc5d0}, 0x1)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/analyzer/batch.go:54 +0x145
github.com/dolthub/go-mysql-server/sql/analyzer.(*Analyzer).analyzeWithSelector(0xc0007dbb60, 0xc00054eb40, {0x29d45f8, 0xc00096d740}, 0x0, 0x271def0)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/analyzer/analyzer.go:376 +0x314
github.com/dolthub/go-mysql-server/sql/analyzer.(*Analyzer).Analyze(...)
/Users/vinairachakonda/go/pkg/mod/github.com/dolthub/go-mysql-server@v0.11.1-0.20211201172932-4744e4cc5d2d/sql/analyzer/analyzer.go:348
...

672: Automatically Generate README File
Doesn't include argument types for functions.
Janky solution to creating Instances of FunctionN.
Might be difficult to edit README now.
668: fix _example/main.go sql.Schema -> sql.PrimaryKeySchema
667: sql/parse: Remove adhoc parsing of certain structures with regexes and custom lexers.
Moves the following to vt/sqlparser:
- SHOW VARIABLES
- SHOW WARNINGS
  Gets rid of unnecessary special handling for SET and SHOW PROCESSLIST.
665: SqlEngine tracks BackgroundThreads
SqlEngine manages the lifecycle of async background threads through a
BackgroundThreads object. Integrators can add goroutines before or
after SqlEngine initialization, but finalizing the engine will terminate
the threads.
Async goroutines are expected to terminate when the context provided by
the thread manager is cancelled. The BackgroundThreads instance uses a
wait group to allow threads to drain gracefully before exiting.
663: Restrict HASH IN conversion to supported operators.
We were converting TupleIn to HashIn for expressions that HashIn could
not evaluate. This change makes explicit which left and right children
we permit for HashIn conversion, and expands HashIn.Eval to evaluate
those expression classes. Added expression unit tests, analyzer unit
tests, query enginetests, and query planner enginetests for more sets of
expressions that we support and do not support.
Additionally, this simplifies the expression evaluation hashing
technique. The new behavior has not been verified with deeply nested
tuple expressions.
661: server,sql/parse: ComQuery, Parse updates for better multi statement support.
657: Fix mysql commands in README
656: Removing unnecessary context creation
651: Fixed IS NOT NULL and IN not being recognized with composite indexes
Fixes #650
649: server: Add option to Config to DisableClientMultiStatements.
648: Added check for conflicting column definitions
Checks for attempt to create primary key on nullable column
Added tests in parse_test.go for new error.
646: faster integral type conversions
we spend up to 10% of sysbench tests in this function
645: Fix Context Creation
642: Track PRIMARY KEY column ordering with a sql.PrimaryKeySchema struct.
The parser exposes PRIMARY KEYs in the TableSpec for CREATE TABLE,
which we pluck before planning. Integrators expected to use the new
sql.PrimaryKeySchema even for keyless tables.
CREATE TABLE LIKE and CREATE TABLE SELECT are handled separately
and need more work to support the same order tracking.
sql.PrimaryKeyAlterableTable now includes PrimaryKeySchema() and Pks()
methods to access the new schema info.
641: Expanded index matches, added many index queries
Besides the obvious addition of a ton of tests (ALL of which make use of indexes, which are enforced by the EXPLAIN statements), this also allows for another scenario to match indexes, although we're still missing a fairly important one. I'll list the scenarios below (assume an index over columns (v1, v2, v3).
The new one we now match: previously expressions had to match the index prefix, but now it has been expanded to require only the first index expression be matched. Therefore, the filter WHERE v1 = 5 AND v3 = 2 is now used by indexes as we can still filter on v1 = 5 using the same principles that partial indexes use. v3 = 2 may not allow for rows to be skipped on the integrator's side when reading from their internal store, but may allow for them to be skipped when returning to the engine (removing the conversion to a sql.Row from the integrator's implementation which should be a fairly substantial speed increase).
This has the added consequence that we can properly combine all ranges that make use of a specific index. That is,
```
(v1 > 1 AND v3 > 1) OR (v1 < 4 AND v2 < 4)
```
will now combine the two ranges (separated by OR). Reducing the prefix limitation has greatly increased the index match possibility. Just to note, the returned indexes are sorted by whether they're an exact match, then by how many expressions are a prefix, then by the index size, and lastly by the index name, so you should always get the "best" index for any given set of expressions.
The above works in the case of a single index existing over the three columns. If (v1, v3, v2) is another index then the above example will match to two different indexes (both being a perfect prefix to an index). There is a way to get around this by taking note of which index has been used previously if multiple indexes share the same columns (or some indexes are subsets/supersets of some other indexes), however this was attempted and abandoned due to the added complexity and time it would take to implement.
Also built off of this, we cannot match
```
(v1 > 1 AND v3 > 1) OR (v2 < 4)
```
As the (v2 < 4) is missing the first index expression (v1), therefore no index is used here (one is used by MySQL, which was tested to confirm that one should be in this case). Again this would be fixed by taking note of which indexes have already been used, as we only need the prefix to be matched one, and then all other sets of expressions just need to be subsets.
Either way, we should now match many more queries than previously.
640: Add utilities needed by Dolt import path
638: Add support for inserting empty value tuples
There was an edge case where if columns were not provided, it required there to be exactly num columns for value tuples, even if there are default values for those columns.
Added a helper function existsNonZeroValues to avoid this.
637: crossjoins converted to inner join if parent filter has suitable join condition
Filter predicates can be pushed into Join nodes under certain conditions:
- Filter is the parent of a CrossJoin
- Predicate expressions cover the two Join node dependency trees
  Filter predicates have to be divided specially to maintain logical equivalence. Only AND conjugates can be split and checked individually for join candidacy. If multiple conjugate trees match the join expression, a new conjugate AND of the subtrees are used as the join condition.
  Nested CrossJoins should be converted to InnerJoins if the parent filter has suitable join comparisons for each.
636: Fixed small issue with used expressions and partial indexes
635: plan,analyzer: Implement a TopRowsNode.
For a query like:
SELECT * FROM table ORDER BY id LIMIT 100 OFFSET 100
we do not need to materialize the entire sorted result. We can use a heap to
compute a streaming TopN and output the results when the child iterator is EOF.
- Add a simple implementation of TopN based on container/heap and
  sql/expression.Sorter.
- Add a TopRowsNode and topRowIter.
- Add a simple analyzer rule to replace a Sort node with a TopRows node when it
  sits directly underneath a Limit() or directly underneath a Limit(Offset()).
  Some care is required to keep sql_calc_found_rows operating as expected.
634: Fixed bug in string -> integer conversions for large integers (caught by dolt)
633: Implement REGEX_REPLACE
Implementation for MySQL regexp_replace function
Includes function tests and engine tests
632: Fixed some expressions not using the largest partial index
631: Fixed String() and WithChildren()
I accidentally clicked merge on last PR.
Fixed String() so that it won't panic on no args
Fixed WithChildren() to compare against current length of args
630: Accept string float values for integer values
629: Added shim to run tests against actual MySQL instance
This will allow GMS tests to be run against an actual local instance of MySQL. A few functions aren’t fully implemented (returning foreign keys for example) but pretty much everything else is.
This will help with both verification of existing tests (as MySQL would be used as the source of truth) and also for writing tests (easy to see what MySQL returns).
For the most part this is just the shim, but I added a single test to show how I’m approaching the GMS tests for mixed operators, so it should be pretty easy to see the utility of the shim.
Full index tests will come in a separate PR.
628: Fix the global variable usage of load data. Re-add LOAD DATA tests
627: add FORMAT() locale
Updated FORMAT() function handling locale with some locale test cases skipped.
626: Add ability to specify precision to current_timestamp function
625: Adding enginetest cases for IP functions
Also change some functions to take interface pointer
622: implement FORMAT() function
Added FORMAT() function for dolt sql
621: Implement IP Functions
Functions Added:
inet_aton
inet_ntoa
inet6_aton
inet6_ntoa
is_ipv4
is_ipv6
is_ipv4_compat
is_ipv4_mapped
Also added tests for each of these functions.
619: UNHEX now returns []byte
Tests have been adjusted to reflect this change
Added another case in function_test for []byte
617: hash in supports tuples
- improve the hash function to handle tuples recursively
- normalize the left side of the IN comparison so GetField is converted to Literal
- analyzer will explicitly fail for HASH IN that should convert doesn't (should it still implicitly fall back?)
  questions:
- what to do about tuple type conversion? should we store the explicit struct of types and validate as we walk the tree while hashing?
- am i missing any edge cases i need to test?
- add max_allowed_packet = 67108864 system variable to limit set size?
615: use hash map for IN equality operator between literal values
614: Fix for joins on columns that are not a prefix to an index
As we now natively support partial indexes in GMS, we had a case where we'd match a single column to a partial index. We would then attempt to use the full index's column expressions which later failed as the columns were not a prefix to the index. Now we enforce that, when we start looking for single column matches, we only find indexes over a single column.
Also, the index builder assumed it would be used if it was created, and therefore could return an range of length 0, so that has been fixed as well. It should only be hit in erroneous cases like the preceding one, but this should prevent panics in other circumstances as well.
612: Added ability to specify custom escape characters for LIKE
There are new test cases for escape syntax.
New parameter for Like Expression.
609: Reduced index range complexity, removed range overlapping
All Ranges now only have a single column expression per range. Ranges no longer overlap. Additional comment changes and whatnot to facilitate both of these changes.
608: TRIM string functions + go.mod tidy
606: Changed some error names
605: Added Support for Full MySQL TRIM Syntax
Parser recognizes keywords like: LEADING, TRAILING, and BOTH.
TRIM function can now remove prefixes.
Fixed LTRIM And RTRIM to not remove all whitespace, and just space characters.
Also added functionality for RIGHT function.
604: Fix Panic in unique key error generation
Poorly written for loop threw an error
602: Add case insensitivity to insert
601: Fix panic in json_unquote for null values
This supercedes #511 and adds tests
599: Remove the Index Editor Sorter and add alias support for Update Join
598: sql/analyzer: pushdown.go: Avoid pushing down projects across BEGIN blocks.
When an UPDATE is within a begin/end block within a trigger, we should avoid
projecting away the full table row.
597: sql/analyzer: pushdown_filters: Avoid pushing down filters below Window nodes.
If a Filter is above a Window node, the Window node will be aggregating and
computing projections based on the rows that it sees. It is not safe to push
the filter below the Window node.
The parser and analyzer currently work in concert (and must work in concert for
Window results to be correct) to keep Filter nodes which do apply to the Window
below the Window.
596: Add Right/left/Cross/Indexed Joins and Implement Table Edit Accumulator
1. Add functionality for updates to work with Indexed Joins
2. Implement a Table Edit Accumulator that writes changes at Close time
3. Fix some undefined test cases
4. Add functionality for Update Join w/ Right,Left,Cross joins
595: persistent session interface
594: sql/plan: Fix join iterators to always Close() their secondary RowIter. Fix Subquery to Dispose its subquery.
585: Changed index interface, support partial keys

vitess

168: Remove glog dependency, reduce log levels, remove all logutilpb usage and support.
167: go/vt/{vttls,tlstest}: Sync from upstream; supports CRL, minimum TLS version configuration.
166: proto: Move .proto files to dolthub/vt subdirectory and package name.
Removes global proto registry conflict with vitessio/vitess, as per #155.
165: Change the prepared statements return value
164: add parsing for SRID
Added parsing for SRID value
SRID value is an integer value that can be upto 2^32-1.
Other fixes:
- Allowed .SQLType() function to evaluate types that are capitalized strings and added tests for it
- Added NULL column option in formatting and added tests for it
162: Adding the SIGNED keyword for numeric type column definitions
SIGNED is the default, so specifying it is allowed in SQL, but is a no-op.
https://dev.mysql.com/doc/refman/8.0/en/numeric-type-syntax.html
Found via: MySQL Grammar Crawler
161: support parsing analyze table t
Adding support to parse analyze statements: https://dev.mysql.com/doc/refman/8.0/en/analyze-table.html
160: return Into value in GetInto() in SelectStatement
This allows to get Into value without doing switch on its types.
159: support parsing of XOR logic operator
Added support for parsing XOR logic operator
158: support SELECT ... INTO syntax
Added supporting of SELECT ... INTO syntax
Added tests for cases with different locations of INTO in SELECT statement
INTO clause can only be used in one place, but it can come in different positions, before [FROM ...] or after the whole SELECT statement but beforing locking. (mysql recommends it after locking options)
INTO OUTFILE ... and INTO DUMPFILE ... are not included yet.
157: add all charset introducers and parsing of charset introducers in LIKE function
- Added all charset introducers supported by MySQL
- Added parsing charset introducer in front of string literal of LIKE function
154: move CURRENT to non-reserved-keyword-list
CURRENT is non reserved keyword, it fixes issue of SELECT * FROM current query failing
Fixes dolthub/dolt#3224
153: Add additional syntax support for CREATE VIEW statement
Added ALGORITHM=..., DEFINER=..., SQL SECURITY DEFINER|INVOKER syntaxes for CREATE VIEW statement.
Fixed definer_opt to parse account_name instead of ID to support parsing of user@locahost syntax.
Moved VIEW (non-reserved keyword) from non-reserved to reserved as non-reserved keywords are used for account_name parsing, so it creates shift/reduce conflict for view.
152: support generated columns
There is no functionality besides parsing.
Fix for: dolthub/dolt#3089
151: Parser support for show create table <table> as of <asof>
Adding support for parsing show create table <table> as of <asof>.
Removing the OnTable field in Show struct that was duplicating the Table field.
Two minor formatting fixes.
150: Parser support for describe table as of and show columns from table as of
Adding parser support for describe <table> as of <asof> and show columns from <table> as of <asof>
149: add db name specification to create/drop TRIGGER statements
148: add db name specification to create/drop procedure statements
Database name is case sensitive but procedure name is not, so TableIdent and ColIdent structs are used for defining db and procedure names, respectively.
147: Have SHOW CREATE PROCEDURE <procedure_name> parse
TODO: need changes in GMS to handle show create procedure logic.
146: /go.mod: require go 1.18
144: Fix of View, Trigger, Procedure position params when used in MySQL special comments
143: Allow Columns to Start with Numbers...kinda
Recognizes identifiers that start with numbers, unless they are immediately after a .
TODO: table names that start with numbers do parse, but there is a regex blocks these from parsing in dolt, so will fix in future PR.
142: Add DISABLE | ENABLE KEYS syntax for ALTER TABLE statement
Added parsing of DISABLE KEYS and ENABLE KEYS options for ALTER TABLE statement
Updated checking Index action keyword to be case insensitive
140: support CHARSET, COLLATE and ENCRYPTION syntax in CREATE DATABASE statement
added parsing of CHARACTER SET, COLLATE and ENCRYPTION options for CREATE DATABASE statement
139: Revert "Merge pull request #137 from dolthub/james/special-comments"
This reverts commit 19d17da876ea71be47f41092d8ab420885b9ba18, reversing
changes made to 0bb5262822024a793f65040a77430ebe0624058e.
137: Handle subquery indexes correctly for special MySQL comments
Fix for: dolthub/dolt#2980
When Tokenizer detects a special MySQL comment, it just creates a new Tokenizer and embeds it in the old one under the specialComment member variable.
This "inner" Tokenizer is then used to handle all the parsing, and its results are just passed to the outer Tokenizer. However, the issue is that the lexer or whatever only reads the Position member variable from the outermost Tokenizer.
So my proposed fix is to just copy over the Position of the inner Tokenizer to the outer one (with an offset to handle the leading /*![12345]. I think a better fix might be to just replace the old Tokenizer with the one we create for specialComment?
136: Initial implementation of fulltext with indexes and DDL
This pr
1. Adds fulltext parsing support for indexes and DDL.
2. Formats the repo
135: Allowing Event Keywords to be used as column name, table name, variable name, etc
134: Get Closer to MySQL's Behavior for Keywords (Reserved and Non-Reserved)
I doubt I'll be able to get all of the keywords working immediately.
This PR focuses on getting as many of the keywords in and working, and documenting what doesn't work yet.
133: reserve found and allow as column safe
132: Add FLUSH as a command and its options
131: Table function support
Add support for table_functions as a new type of table factor in the dolthub/vitess SQL grammar.
130: add PROCESSLIST to non-reserved-keyword list
129: Adding scripts to test parsing behavior of MySQL
128: /go/bucketpool: Optimized findBucket
127: Add testcases for exercising keywords and remove some create user tests
This PR:
1. Allows the COMMENT_KEYWORD to be alias by selects/
2. Removes some usages of aliased keywords in the select queries
3. Adds skipped tests that exercises the validity of reserved and non reserved keywords through a series of tests
126: go/mysql: Add buffering to prepared statement result sets
121: Add show like status
120: Named Windows
Add yacc syntax for window names. WindowDef is used with either its Name
field in a WINDOW clause, or NameRef in an OVER clause. NameRef is OVER
is not mandatory, but Name in WINDOW is.
This PR adds precedence rules to avoid shift/reduce conflicts between
the WINDOW name field, which can be any string, and RANGE, PARTITION,
ROWS, and RANGE should be prioritized for other rules if they appear as
the first token in a window_spec.
119: partitionBy and overBy optional, one mandatory with frame def
118: Added remaining privileges
This adds the remaining privileges.
117: recursive CTEs
116: tools.go hack.
115: go.mod: Bump to go 1.17.
114: Remove a bunch of unused stuff.
113: .github,go: Cleanup workflows. Fix all tests to work under -race. Fix fragile prototext assertions in tests.
112: go,go/vt/proto: Regenerate pb message implementations, upgrade to APIv2.
111: go/vt/sqlparser: Get rid of partially parsed statements and {,ddl_}skip_to_end.
110: add serial supoort
add serial support https://dev.mysql.com/doc/refman/8.0/en/numeric-type-syntax.html#:~:text=SERIAL
109: go/mysql: Make all tests pass on macOS.
Bumps expected TLS version from 1.2 to 1.3.
Adds some string matching for strerror strings that are different between Linux
and macOS.
108: add support for constraint named unique
This is an enhancement for #95
According to https://dev.mysql.com/doc/refman/8.0/en/create-table.html, there will be a format like constraint namedx unique (full_name) in create table statement.
107: go/vt/sqlparser: Remove PartialDDL support.
There's no need to support partial DDL in dolt.
104: Window function frames, extent, and bounds
Adds parsing capabilities for window frames, including one sided frames, two sided frames, RANGE value frames, integral and interval based frames, and errors for semantically incorect combinations of frame options. Only value based intervals are currently supported. No prepared statement support for expression bounds.
103: go/mysql: conn: Add LoadInfile method which returns a ReadCloser for reading the remote file from a client.
We will migrate the implementation in go-mysql-server from HandleLoadDataLocalQuery to this interface. This keeps the responsibility for writing the file and the likes out of the vitess/conn layer.
102: Added GRANT, REVOKE, and the remainder of CREATE USER
101: go/vt/sqlparser: Parse KILL {CONNECTION,QUERY} statements.
100: go/vt/sqlparser: Extend the parser for correctly parsing some extended SHOW syntax.
- SHOW COUNT(*) {WARNINGS, ERRORS}
- SHOW {WARNINGS,ERRORS} LIMIT ...[, ...]
- SHOW {,SESSION,GLOBAL} VARIABLES {LIKE,WHERE} ...
- SHOW FULL PROCESSLIST
99: Added more statements for user handling
Added RENAME USER, CREATE ROLE, DROP ROLE, SHOW GRANTS, and a portion of CREATE USER. The rest of CREATE USER, along with GRANT and REVOKE will come in the next PR.
97: Added proper syntax parsing for account names, along with DROP USER
This PR primarily implements the changes necessary for account names, which required edits in the parser itself. DROP USER was implemented to test the name variations, so refer to the tests in parse_test.go for how account names work (and to see why the changes were necessary). All parser tests were validated against a MySQL 8.0 instance.
In a nutshell, account names take the form user@host, and both user and host must be separate strings. The parser treated @ as a standard letter, and we used regex matching in GMS and Dolt to catch "invalid" names (@@system_var is valid but what@@ is invalid). My first pass was to treat the entire account name as a string and split it using a regex, however I could not get it to pass all of the parser tests. This was the only way I could make it work with all tests passing.
Notes:
- The other statements will come in a different PR, which will be straightforward in comparison.
- DROP USER ''; looks like weird syntax, but an empty name means that all names match. In this case, you can assign some dbs/tables to be available to everyone.
- Following the above, DROP USER ``; is valid, but failed as we returned an error on empty quoted identifiers. MySQL allows empty quoted identifiers to parse, but rejects them as invalid names depending on the query. This has been fixed.
- Added ast_permissions.go to put all of the statements dealing with users, grants, etc. For me, ast.go operates very slowly and occasionally hangs for a few minutes, so I'm putting my new statements in a new file for now. I doubt you'll have pushback, but if so I can merge the file back into ast.go when I'm done.
96: Better support for MySQL MULTI_STATEMENTS.
95: Added support for CONSTRAINT UNIQUE KEY syntax in CREATE TABLE
94: add Null to column definition
create table t (a int primary key) succeeds
create table t (a int not null primary key) succeeds
create table t (a int null primary key) fails
Create new BoolVal field for ColumnDefinition to more easily check for attempts to create nullable columns with primary key.
Also allows people to do create table t(a int null not null ...) if they want.
93: go/mysql/conn.go: Add a way to disable client_multi_statements handling on a per-connection basis.
Conn's handling of CLIENT_MULTI_STATEMENTS is currently wrong for packets that
contain statements which can include other statements (e.g., CREATE TRIGGER).
Add a way for an integrator to disable it for now.
92: parser can detect insert into () values <tuple_list>
added new case for insert_data
should be able to correctly parse
insert into test_tb values ()
insert into test_tb () values ()
insert into test_tb values (), (), ...
insert into test_tb () values (), (), ...
91: added FORMAT keyword
added format expression keyword
90: Noop behavior for DROP TABLE CASCASE/RESTRICT
89: Added Test Cases for LIKE ... ESCAPE ...
88: Add TRIM functionality
Changed sql.y to recognize keywords TRIM, LEADING, TRAILING, and BOTH
85: can use status w/o quotes for insert statement
re: dolthub/dolt#2195

Closed Issues

1050: Complex query with ABS, !=, REGEXP, and CONVERT fails.
950: README suggests installing via now depreciated "go get"
989: willing to maintaion sql standard schema support?
973: no index record in information_schema.statistics
808: FilteredTable uses wrong schema with alias
911: _example Failure to output
916: RightIndexedJoin commutativity correctness
894: when is version 1.0 api standard set?
900: Does go-mysql-server support create database and table from SQL file?
901: ERROR 1105 (HY000): Error: index already exists
888: The _example/main.go compile error.
815: Index error lost in parent call
790: String variables aren't evaluated correctly in where clauses
799: Aggregate Partition Window Rows beyond 127
146: regex-metrics can't be initialized
171: Two foreign keys in same CREATE TABLE statements triggers error
355: Question/Feature Request: How can I increase the parallelism of expression evaluation?
654: fatal error: concurrent map write read
705: error: can't modify database databaseProvider
709: create table with unnamed key got error ERROR 1105 (HY000): Error: index already exists
191: Support for VALUES in INSERT ... ON DUPLICATE KEY UPDATE
250: unsupported syntax: create database mapping
313: How can I custom my log.
481: Anyone working on the replication?
703: undefined: sql.NewPrimaryKeySchema
662: Automatically generate function docs
664: _example/main.go unable to compile
653: HASH IN fails on arithmetic, function nodes
618: Unhex function returns a string, when it should return []byte
650: IS NOT NULL and IN are causing composite index to be ignored
643: pls dont rush the API 1.0 but do think properly and optimize for performance
172: Support for prepared statements
179: Combine index lookups on different columns of multi-column indexes with different operators
603: Panic when using memory implemenation
584: Proxy support?
155: Vitess protocol buffer namespace conflict

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

June 2022 Release

Merged PRs

go-mysql-server

vitess

Closed Issues