Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ttljob: when a range intersects two tables, the job fails #81208

Closed
otan opened this issue May 12, 2022 · 0 comments
Closed

ttljob: when a range intersects two tables, the job fails #81208

otan opened this issue May 12, 2022 · 0 comments
Assignees
Labels
A-row-level-ttl branch-release-22.1 Used to mark GA and release blockers, technical advisories, and bugs for 22.1 C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. deprecated-branch-release-22.1.0 GA-blocker T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)

Comments

@otan
Copy link
Contributor

otan commented May 12, 2022

Consider the following setup:

CREATE TABLE tbl (
	id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
	text TEXT
) WITH (ttl_expire_after = '30 days');
ALTER TABLE tbl SPLIT AT VALUES (gen_random_uuid());
CREATE TABLE tbl2 (id INT PRIMARY KEY);
ALTER TABLE tbl2 SPLIT AT VALUES (1);

In this case, there may be a range that is valid between /tbl1/gen_random_uuid() and /tbl2/1. If this is the case, a TTL job may fail with a error decoding X bytes message. The impact here is that TTL does not succeed and rows do not get cleaned up (foreground traffic still occurs).

In practice, this would only happen if:

  • the table is sufficiently large.
  • there are multiple tables in the database
  • usually only the last range on the table is affected - other rows are TTL'd

Jira issue: CRDB-15299

@otan otan added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. GA-blocker A-row-level-ttl branch-release-22.1 Used to mark GA and release blockers, technical advisories, and bugs for 22.1 deprecated-branch-release-22.1.0 and removed C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) labels May 12, 2022
@otan otan self-assigned this May 12, 2022
@blathers-crl blathers-crl bot added the T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) label May 12, 2022
craig bot pushed a commit that referenced this issue May 14, 2022
79623: sql: add virtual indexes to crdb_internal.cluster_locks virtual table  r=AlexTalks a=AlexTalks

This change adds virtual indexes on the `table_id`, `database_name`, and
`table_name` columns of the `crdb_internal.cluster_locks` virtual table,
so that when queried, the `kv.Batch`es with `QueryLocksRequest`s can be
constrained to query only specific tables or databases. This allows the
requests to be much more limited, rather than needing to request all of
the ranges that comprise the key spans of all tables accessible by the
user.

Release note (sql change): Improved query performance for
`crdb_internal.cluster_locks` when issued with constraints in the WHERE
clause on `table_id`, `database_name`, or `table_name` columns.

Depends on #77876, #80422

80832: kvserver: IsCompleteTransaction might panic with certain batch sequences r=shralex a=shralex

It's unclear how this panic happened. One possibility is that EntTxn had a negative sequence number. Another hypothesis is that ba.Requests was concurrently mutated due to a data race. This happened once, so for now adding more info to the panic.

Release note: None

Jira issue: https://cockroachlabs.atlassian.net/browse/CRDB-14627

81190: roachtest: update activerecord adapter to v6.1.10 r=rafiss a=ecwall

refs #67893
refs #80777

This version correctly disables supports_expression_index to
prevent `ON CONFLICT expression` from appearing in generated
SQL statements.

Release note: None

81193: storage: upgrade to pebblev2 table format r=jbowens,nicktrav a=erikgrinaker

Resubmit of #76780, which was partially reverted for 22.1.

---

The `Pebblev2` SSTable format adds support for range keys. Add two new
cluster versions to provide the upgrade path - the first version for
bumping the store, the second for use as a feature gate.

Release note: None

81207: ttljob: fix a range edge case r=rafiss a=otan

See individual commits for details.

Refs: #81208

Co-authored-by: Alex Sarkesian <sarkesian@cockroachlabs.com>
Co-authored-by: shralex <shralex@gmail.com>
Co-authored-by: Evan Wall <wall@cockroachlabs.com>
Co-authored-by: Erik Grinaker <grinaker@cockroachlabs.com>
Co-authored-by: Oliver Tan <otan@cockroachlabs.com>
@otan otan closed this as completed May 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-row-level-ttl branch-release-22.1 Used to mark GA and release blockers, technical advisories, and bugs for 22.1 C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. deprecated-branch-release-22.1.0 GA-blocker T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Projects
None yet
Development

No branches or pull requests

2 participants