clusterversion,storage: support Pebblev2 table format #76780
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This change is |
erikgrinaker
approved these changes
Feb 18, 2022
nicktrav
force-pushed
the
nickt.range-key-cluster-versions
branch
from
c1ccf80
to
fcd6739
Compare
29 tasks
nicktrav
force-pushed
the
nickt.range-key-cluster-versions
branch
3 times, most recently
from
0089bca
to
8a1fd47
Compare
The `Pebblev2` SSTable format adds support for range keys. Add two new cluster versions to provide the upgrade path - the first version for bumping the store, the second for use as a feature gate. Rework the table format inference for new SSTable writers to take a more conservative approach. By default, assume version `RocksDBv2`, and only bump up to a newer version if a cluster supports it. Previously a newer version was assumed and bumped down if the cluster didn't support it. Update `storage.MakeBackupSSTWriter` to take in a context and cluster settings which can be used to infer which table version to use. Enable range keys for backups and ingest, if the cluster supports it. Release note: None
nicktrav
force-pushed
the
nickt.range-key-cluster-versions
branch
from
8a1fd47
to
9ecc619
Compare
|
TFTR! bors r=erikgrinaker |
craig bot
pushed a commit
that referenced
this pull request
Feb 21, 2022
76780: clusterversion,storage: support Pebblev2 table format r=erikgrinaker a=nicktrav The `Pebblev2` SSTable format adds support for range keys. Add two new cluster versions to provide the upgrade path - the first version for bumping the store, the second for use as a feature gate. Rework the table format inference for new SSTable writers to take a more conservative approach. By default, assume version `RocksDBv2`, and only bump up to a newer version if a cluster supports it. Previously a newer version was assumed and bumped down if the cluster didn't support it. Update `storage.MakeBackupSSTWriter` to take in a context and cluster settings which can be used to infer which table version to use. Enable range keys for backups and ingest, if the cluster supports it. Release note: None Co-authored-by: Nick Travers <travers@cockroachlabs.com>
|
Build failed: |
|
bors retry |
|
Build succeeded: |
craig bot
pushed a commit
that referenced
this pull request
May 14, 2022
79623: sql: add virtual indexes to crdb_internal.cluster_locks virtual table r=AlexTalks a=AlexTalks This change adds virtual indexes on the `table_id`, `database_name`, and `table_name` columns of the `crdb_internal.cluster_locks` virtual table, so that when queried, the `kv.Batch`es with `QueryLocksRequest`s can be constrained to query only specific tables or databases. This allows the requests to be much more limited, rather than needing to request all of the ranges that comprise the key spans of all tables accessible by the user. Release note (sql change): Improved query performance for `crdb_internal.cluster_locks` when issued with constraints in the WHERE clause on `table_id`, `database_name`, or `table_name` columns. Depends on #77876, #80422 80832: kvserver: IsCompleteTransaction might panic with certain batch sequences r=shralex a=shralex It's unclear how this panic happened. One possibility is that EntTxn had a negative sequence number. Another hypothesis is that ba.Requests was concurrently mutated due to a data race. This happened once, so for now adding more info to the panic. Release note: None Jira issue: https://cockroachlabs.atlassian.net/browse/CRDB-14627 81190: roachtest: update activerecord adapter to v6.1.10 r=rafiss a=ecwall refs #67893 refs #80777 This version correctly disables supports_expression_index to prevent `ON CONFLICT expression` from appearing in generated SQL statements. Release note: None 81193: storage: upgrade to pebblev2 table format r=jbowens,nicktrav a=erikgrinaker Resubmit of #76780, which was partially reverted for 22.1. --- The `Pebblev2` SSTable format adds support for range keys. Add two new cluster versions to provide the upgrade path - the first version for bumping the store, the second for use as a feature gate. Release note: None 81207: ttljob: fix a range edge case r=rafiss a=otan See individual commits for details. Refs: #81208 Co-authored-by: Alex Sarkesian <sarkesian@cockroachlabs.com> Co-authored-by: shralex <shralex@gmail.com> Co-authored-by: Evan Wall <wall@cockroachlabs.com> Co-authored-by: Erik Grinaker <grinaker@cockroachlabs.com> Co-authored-by: Oliver Tan <otan@cockroachlabs.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The
Pebblev2SSTable format adds support for range keys. Add two newcluster versions to provide the upgrade path - the first version for
bumping the store, the second for use as a feature gate.
Rework the table format inference for new SSTable writers to take a more
conservative approach. By default, assume version
RocksDBv2, and onlybump up to a newer version if a cluster supports it. Previously a newer
version was assumed and bumped down if the cluster didn't support it.
Update
storage.MakeBackupSSTWriterto take in a context and clustersettings which can be used to infer which table version to use. Enable
range keys for backups and ingest, if the cluster supports it.
Release note: None