-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[YSQL] ERROR: The column already exists when trying to add column after dropping with drop domain cascade #22874
Labels
2.20 Backport Required
2024.1 Backport Required
area/docdb
YugabyteDB core features
area/ysql
Yugabyte SQL (YSQL)
kind/bug
This issue is a bug
priority/medium
Medium priority issue
Comments
archit-rastogi
added
area/ysql
Yugabyte SQL (YSQL)
status/awaiting-triage
Issue awaiting triage
labels
Jun 14, 2024
yugabyte-ci
added
kind/bug
This issue is a bug
priority/medium
Medium priority issue
labels
Jun 14, 2024
fizaaluthra
added a commit
that referenced
this issue
Jun 27, 2024
Summary: Currently, when an object that a column depends on is dropped, the drop isn't correctly cascaded to the column in the DocDB table itself. Example: ``` yugabyte=# CREATE TABLE test_dropcolumn(a int, b int, c int); CREATE TABLE yugabyte=# CREATE TYPE test_dropcolumn_type AS (a int, b int); CREATE TYPE yugabyte=# ALTER TABLE test_dropcolumn ADD COLUMN d test_dropcolumn_type; ALTER TABLE yugabyte=# DROP TYPE test_dropcolumn_type CASCADE; NOTICE: drop cascades to column t of table person DROP TYPE yugabyte=# ALTER TABLE test_dropcolumn ADD COLUMN d int; ERROR: The column already exists: d ``` Although the PG metadata for the column is dropped, the column itself is not dropped in DocDB. This leads to an inconsistency between DocDB and PG. This diff fixes the issue by adding a YB drop for the column inside `doDeletion`. Note: this function is also used during `ALTER TABLE... DROP COLUMN`; however, alter table commands execute YB schema changes separately (see `YBCPrepareAlterTableCmd` and `YBCExecAlterTable`). So, an additional flag is added to skip the YB column drop in `doDeletion` in case it is being invoked by the alter table flow. Jira: DB-11776 Test Plan: yb_alter_table Reviewers: myang Reviewed By: myang Subscribers: yql Differential Revision: https://phorge.dev.yugabyte.com/D35866
jasonyb
pushed a commit
that referenced
this issue
Jun 28, 2024
Summary: 9c637e2 [PLAT-14429]: Modify Troubleshooting Platform registration workflow in YBA 0a1406d [PLAT-14098]: Updating yb.runtime_conf_ui.tag_filter with appropriate tags value does not display the flags accordingly 70a87f9 [PLAT-13605][PLAT-13609]: Edit Volume controls and storage type in FULL_MOVE but not in case of UPDATE 26fbfe0 [PLAT-14515][UI] Clicking preview doesn't show the correct info and clears up the data provided while setting up the ysql_ident or ysql_hba multiline flags.- [PLAT-14514] [PLAT-14513] a07946b [#18233] Initial commit for yugabyted unit test framework. b2e8ee7 [#22842] docdb: Improve usability of stack trace tracking endpoints 508f26e [docs] Added RN 2.20.2.3-b2 (#23042) 214d44a [#22935] YSQL: Use db oid in the tserver's sequence cache entry key c47b2d9 [#22802] YSQL: Avoid renaming DocDb tables during legacy rewrite Excluded: 7c8343d [#22874] YSQL: Fix cascaded drops on columns 58c8d4e [#23046] xCluster: Remove ns_replication from the code a70681d [#22816] YSQL: Bug fixes for replication connections in ysql connection manager b239e07 Doc upgrade versions (#22988) Excluded: ec76062 [#16408] YSQL: Split TestPgRegressIndex.testPgRegressIndex Test Plan: Jenkins: rebase: pg15-cherrypicks Reviewers: jason, tfoucher Subscribers: yql Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D36247
fizaaluthra
added a commit
that referenced
this issue
Jul 2, 2024
Summary: Cherry-pick YB master commit 7c8343d titled [#22874] YSQL: Fix cascaded drops on columns and committed 2024-06-27T12:41:11-04:00 into YB pg15. Upstream PG commit 1df5875d39383b3981b804666ee1f4b0ff65943f changes the `ATExecDropColumn` to use `performMultipleDeletions` (instead of `performDeletion`). YB master commit 7c8343d changes the `performDeletion` call to use `YB_SKIP_YB_DROP_COLUMN` as the flag parameter. Resolve the conflict by using `YB_SKIP_YB_DROP_COLUMN` as the flag parameter in the performMultipleDeletions call so that we skip YB column drops in `doDeletion`. Jira: DB-11776 Test Plan: ./yb_build.sh --java-test 'org.yb.pgsql.TestPgRegressTable' ./yb_build.sh --java-test 'org.yb.pgsql.TestPgRegressTypesUDT' Manually verified the test introduced by PG commit 1df5875d39383b3981b804666ee1f4b0ff65943f: ``` -- check that dropping a column takes with it any partitioned indexes -- depending on it. create table parted_index_col_drop(a int, b int, c int) partition by list (a); create table parted_index_col_drop1 partition of parted_index_col_drop for values in (1) partition by list (a); -- leave this partition without children. create table parted_index_col_drop2 partition of parted_index_col_drop for values in (2) partition by list (a); create table parted_index_col_drop11 partition of parted_index_col_drop1 for values in (1); create index on parted_index_col_drop (b); create index on parted_index_col_drop (c); create index on parted_index_col_drop (b, c); alter table parted_index_col_drop drop column c; \d parted_index_col_drop \d parted_index_col_drop1 \d parted_index_col_drop2 \d parted_index_col_drop11 drop table parted_index_col_drop; ``` Reviewers: jason, tfoucher Reviewed By: jason Subscribers: yql Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D36309
fizaaluthra
added a commit
that referenced
this issue
Jul 12, 2024
Summary: Commit 7c8343d / D35866 summary: Currently, when an object that a column depends on is dropped, the drop isn't correctly cascaded to the column in the DocDB table itself. Example: ``` yugabyte=# CREATE TABLE test_dropcolumn(a int, b int, c int); CREATE TABLE yugabyte=# CREATE TYPE test_dropcolumn_type AS (a int, b int); CREATE TYPE yugabyte=# ALTER TABLE test_dropcolumn ADD COLUMN d test_dropcolumn_type; ALTER TABLE yugabyte=# DROP TYPE test_dropcolumn_type CASCADE; NOTICE: drop cascades to column t of table person DROP TYPE yugabyte=# ALTER TABLE test_dropcolumn ADD COLUMN d int; ERROR: The column already exists: d ``` Although the PG metadata for the column is dropped, the column itself is not dropped in DocDB. This leads to an inconsistency between DocDB and PG. This diff fixes the issue by adding a YB drop for the column inside `doDeletion`. Note: this function is also used during `ALTER TABLE... DROP COLUMN`; however, alter table commands execute YB schema changes separately (see `YBCPrepareAlterTableCmd` and `YBCExecAlterTable`). So, an additional flag is added to skip the YB column drop in `doDeletion` in case it is being invoked by the alter table flow. Jira: DB-11776 Commit b81dbaa / D36329 summary: The test became flaky after commit 7c8343d fixed the logic for cascaded drops on columns. The test yb_feature_hash_types creates a domain and creates a table with a key column that has the domain as its type. However, since we currently don't support key column drops (#22902), the test fails when the cleanUpCustomEntities() function in BasePgSQLTest.java attempts to drop the domain (that has the key column as a dependant object) during cleanup. Moreover, the test is flaky because sometimes the table may be dropped before the domain, as the order of the drops depends on the pg_depend scan output, which isn't deterministic because pg_depend doesn't have a primary key and internally uses ybctid. Fix the test flakiness by dropping the table in yb_feature_hash_types before the test clean up logic. Jira: DB-12029 Original commits: 7c8343d / D35866 b81dbaa / D36329 Test Plan: yb_alter_table Reviewers: myang Reviewed By: myang Subscribers: yql Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D36286
fizaaluthra
added a commit
that referenced
this issue
Jul 12, 2024
Summary: Commit 7c8343d / D35866 summary: Currently, when an object that a column depends on is dropped, the drop isn't correctly cascaded to the column in the DocDB table itself. Example: yugabyte=# CREATE TABLE test_dropcolumn(a int, b int, c int); CREATE TABLE yugabyte=# CREATE TYPE test_dropcolumn_type AS (a int, b int); CREATE TYPE yugabyte=# ALTER TABLE test_dropcolumn ADD COLUMN d test_dropcolumn_type; ALTER TABLE yugabyte=# DROP TYPE test_dropcolumn_type CASCADE; NOTICE: drop cascades to column t of table person DROP TYPE yugabyte=# ALTER TABLE test_dropcolumn ADD COLUMN d int; ERROR: The column already exists: d Although the PG metadata for the column is dropped, the column itself is not dropped in DocDB. This leads to an inconsistency between DocDB and PG. This diff fixes the issue by adding a YB drop for the column inside doDeletion. Note: this function is also used during ALTER TABLE... DROP COLUMN; however, alter table commands execute YB schema changes separately (see YBCPrepareAlterTableCmd and YBCExecAlterTable). So, an additional flag is added to skip the YB column drop in doDeletion in case it is being invoked by the alter table flow. Jira: DB-11776 Commit b81dbaa / D36329 summary: The test became flaky after commit 7c8343d fixed the logic for cascaded drops on columns. The test yb_feature_hash_types creates a domain and creates a table with a key column that has the domain as its type. However, since we currently don't support key column drops (#22902), the test fails when the cleanUpCustomEntities() function in BasePgSQLTest.java attempts to drop the domain (that has the key column as a dependant object) during cleanup. Moreover, the test is flaky because sometimes the table may be dropped before the domain, as the order of the drops depends on the pg_depend scan output, which isn't deterministic because pg_depend doesn't have a primary key and internally uses ybctid. Fix the test flakiness by dropping the table in yb_feature_hash_types before the test clean up logic. Jira: DB-12029 Original commits: 7c8343d / D35866 b81dbaa / D36329 Test Plan: yb_alter_table Reviewers: myang Reviewed By: myang Subscribers: yql Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D36289
fizaaluthra
added a commit
that referenced
this issue
Jul 12, 2024
Summary: Commit 7c8343d / D35866 summary: Currently, when an object that a column depends on is dropped, the drop isn't correctly cascaded to the column in the DocDB table itself. Example: ``` yugabyte=# CREATE TABLE test_dropcolumn(a int, b int, c int); CREATE TABLE yugabyte=# CREATE TYPE test_dropcolumn_type AS (a int, b int); CREATE TYPE yugabyte=# ALTER TABLE test_dropcolumn ADD COLUMN d test_dropcolumn_type; ALTER TABLE yugabyte=# DROP TYPE test_dropcolumn_type CASCADE; NOTICE: drop cascades to column t of table person DROP TYPE yugabyte=# ALTER TABLE test_dropcolumn ADD COLUMN d int; ERROR: The column already exists: d ``` Although the PG metadata for the column is dropped, the column itself is not dropped in DocDB. This leads to an inconsistency between DocDB and PG. This diff fixes the issue by adding a YB drop for the column inside `doDeletion`. Note: this function is also used during `ALTER TABLE... DROP COLUMN`; however, alter table commands execute YB schema changes separately (see `YBCPrepareAlterTableCmd` and `YBCExecAlterTable`). So, an additional flag is added to skip the YB column drop in `doDeletion` in case it is being invoked by the alter table flow. Jira: DB-11776 Commit b81dbaa / D36329 summary: The test became flaky after commit 7c8343d fixed the logic for cascaded drops on columns. The test yb_feature_hash_types creates a domain and creates a table with a key column that has the domain as its type. However, since we currently don't support key column drops (#22902), the test fails when the cleanUpCustomEntities() function in BasePgSQLTest.java attempts to drop the domain (that has the key column as a dependant object) during cleanup. Moreover, the test is flaky because sometimes the table may be dropped before the domain, as the order of the drops depends on the pg_depend scan output, which isn't deterministic because pg_depend doesn't have a primary key and internally uses ybctid. Fix the test flakiness by dropping the table in yb_feature_hash_types before the test clean up logic. Jira: DB-12029 Original commits: 7c8343d / D35866 b81dbaa / D36329 Test Plan: yb_alter_table Reviewers: myang Reviewed By: myang Subscribers: yql Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D36290
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
2.20 Backport Required
2024.1 Backport Required
area/docdb
YugabyteDB core features
area/ysql
Yugabyte SQL (YSQL)
kind/bug
This issue is a bug
priority/medium
Medium priority issue
Jira Link: DB-11776
Description
Step to reproduce:
Example:
Issue Type
kind/bug
Warning: Please confirm that this issue does not contain any sensitive information
The text was updated successfully, but these errors were encountered: