sql: support virtual table indexes with generators #80422
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This change is |
AlexTalks
changed the title
AlexTalks
force-pushed
the
virtual_index_w_generator
branch
from
1a95525
to
1974f8f
Compare
While previously virtual schema tables had support for defining tables with a `populate` function, which eagerly loads all rows, or a `generator` function, which lazy loads each row when called (possibly running in a worker Goroutine), and also had support for virtual indexes which would have their own `populate` functions, there was a subtle lack of support for using virtual indexes with virtual tables that used a `generator`, since the virtual index constraint logic would fall back to a (possibly undefined) `populate` function in several cases. This change fixes the nil pointer exception that could occur if using virtual indexes with a table using a `generator`, and validates that the virtual index is supported prior to use. Release note: None
AlexTalks
force-pushed
the
virtual_index_w_generator
branch
from
1974f8f
to
622be5c
Compare
This was referenced Apr 25, 2022
jordanlewis
approved these changes
May 5, 2022
There was a problem hiding this comment.
Thanks for the fix, and I appreciate your patience on the review!
Reviewed 1 of 1 files at r1, all commit messages.
Reviewable status:complete! 1 of 0 LGTMs obtained (waiting on @rafiss)
|
bors r+ |
|
Build succeeded: |
craig bot
pushed a commit
that referenced
this pull request
May 14, 2022
79623: sql: add virtual indexes to crdb_internal.cluster_locks virtual table r=AlexTalks a=AlexTalks This change adds virtual indexes on the `table_id`, `database_name`, and `table_name` columns of the `crdb_internal.cluster_locks` virtual table, so that when queried, the `kv.Batch`es with `QueryLocksRequest`s can be constrained to query only specific tables or databases. This allows the requests to be much more limited, rather than needing to request all of the ranges that comprise the key spans of all tables accessible by the user. Release note (sql change): Improved query performance for `crdb_internal.cluster_locks` when issued with constraints in the WHERE clause on `table_id`, `database_name`, or `table_name` columns. Depends on #77876, #80422 80832: kvserver: IsCompleteTransaction might panic with certain batch sequences r=shralex a=shralex It's unclear how this panic happened. One possibility is that EntTxn had a negative sequence number. Another hypothesis is that ba.Requests was concurrently mutated due to a data race. This happened once, so for now adding more info to the panic. Release note: None Jira issue: https://cockroachlabs.atlassian.net/browse/CRDB-14627 81190: roachtest: update activerecord adapter to v6.1.10 r=rafiss a=ecwall refs #67893 refs #80777 This version correctly disables supports_expression_index to prevent `ON CONFLICT expression` from appearing in generated SQL statements. Release note: None 81193: storage: upgrade to pebblev2 table format r=jbowens,nicktrav a=erikgrinaker Resubmit of #76780, which was partially reverted for 22.1. --- The `Pebblev2` SSTable format adds support for range keys. Add two new cluster versions to provide the upgrade path - the first version for bumping the store, the second for use as a feature gate. Release note: None 81207: ttljob: fix a range edge case r=rafiss a=otan See individual commits for details. Refs: #81208 Co-authored-by: Alex Sarkesian <sarkesian@cockroachlabs.com> Co-authored-by: shralex <shralex@gmail.com> Co-authored-by: Evan Wall <wall@cockroachlabs.com> Co-authored-by: Erik Grinaker <grinaker@cockroachlabs.com> Co-authored-by: Oliver Tan <otan@cockroachlabs.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
While previously virtual schema tables had support for defining tables
with a
populatefunction, which eagerly loads all rows, or ageneratorfunction, which lazy loads each row when called (possiblyrunning in a worker Goroutine), and also had support for virtual indexes
which would have their own
populatefunctions, there was a subtle lackof support for using virtual indexes with virtual tables that used a
generator, since the virtual index constraint logic would fall back toa (possibly undefined)
populatefunction in several cases. This changefixes the nil pointer exception that could occur if using virtual
indexes with a table using a
generator, and validates that the virtualindex is supported prior to use.
Release Note: None
Release Justification: Bug fix