Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DocDB] Backward scans: make pggate be aware of fast backward scan capability #22937

Closed
1 task done
arybochkin opened this issue Jun 20, 2024 · 0 comments
Closed
1 task done

[DocDB] Backward scans: make pggate be aware of fast backward scan capability #22937

arybochkin opened this issue Jun 20, 2024 · 0 comments
Assignees
@arybochkin
Labels
area/docdb YugabyteDB core features kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue

Comments

@arybochkin
Copy link
Contributor

arybochkin commented Jun 20, 2024 •
edited
Loading

Jira Link: DB-11853

Description

It is required to make pggate be aware of fast backward scan capability to be able to correctly identify the cost of backward scans in Cost Based Optimizer (#22370).

Issue Type

kind/enhancement

Warning: Please confirm that this issue does not contain any sensitive information

  • I confirm this issue does not contain any sensitive information.
@arybochkin arybochkin added area/docdb YugabyteDB core features status/awaiting-triage Issue awaiting triage labels Jun 20, 2024
@yugabyte-ci yugabyte-ci added kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue labels Jun 20, 2024
@yugabyte-ci yugabyte-ci removed the status/awaiting-triage Issue awaiting triage label Jun 26, 2024
@arybochkin arybochkin changed the title [DocDB] Backward scans: move ysql_use_flat_doc_reader and use_fast_backward_scan to GUC [DocDB] Backward scans: move use_fast_backward_scan to GUC Jun 28, 2024
@arybochkin arybochkin changed the title [DocDB] Backward scans: move use_fast_backward_scan to GUC [DocDB] Backward scans: make pggate be aware of fast backward scan capability Jul 1, 2024
arybochkin added a commit that referenced this issue Jul 15, 2024
@arybochkin
[#22937] docdb: Backward scans: make pggate be aware of fast backward…
1773ae2 
… scan capability

Summary:
This is a first part of CBO-related changes for backward scans, which is not directly related to
the cost calculation. But the aim of the change is to make pggate be aware of fast backward scan
capability to be able to correctly identify the cost of backward scans in the CBO.
Refer to #22370 for the details.

For this purpose the flag is moved to common_flags.cc and YBCPgGFlagsAccessor is updated
to be able to get the value of the flag. Additionally the flag is mark as `NON_RUNTIME` to prevent
any confusion from a user side as PG flags does not support runtime change. Such change is
acceptable as it is not expected the flag change in future.

**Upgrade/Rollback safety:**
In the worst case, when the node is not yet upgraded, the planner/optimizer will not be aware of
the fast backward scan capability and as a result would be choosing a different execution path
without fast backward scan optimization avoiding possible performance gain. On the other hand,
if the planner/optimizer already aware of the capability (and it's on), but remote yb-tserver is
not yet upgraded, then it may result in a slower query but with no correctness impact.

Jira: DB-11853

Test Plan: Jenkins

Reviewers: sergei, amartsinchyk, rthallam

Reviewed By: amartsinchyk

Subscribers: hsunder, ybase, yql

Differential Revision: https://phorge.dev.yugabyte.com/D36245
jasonyb pushed a commit that referenced this issue Jul 17, 2024
@yugabyte-ci @jaki
[BACKPORT pg15-cherrypicks] all: Bulk port from master - 55
6c7ea34 
Summary:
 cc63aaf [docs] updates to diagnostics report page (#23162)
 Excluded: 1773ae2 [#22937] docdb: Backward scans: make pggate be aware of fast backward scan capability
 39c6228 [PLAT-12732] toggle tls and cert rotation v2 apis
 d7cf125 [PLAT-14539][xCluster] need_bootstrap API does not work for old universes
 65232ff [PLAT-14654]: Volume Size does not increase when ULTRA storage type is selected
 4b39933 [PLAT-14606]: Disable options to ensure shrinking the RF is not permitted  edit universe scenario
 98d3fed [#23182] YSQL: Fix upgrade test failure when using 2.20.3.1 snapshot
 Excluded: 19ab966 [#22902] YSQL: Add support for dropping a key column
 835e30d [#22479] docdb: Pass epoch through DB cloning calls
 3273e9b [#21789] docdb: Add tablet splitting support for clone
 52f7e79 [#23064] YSQL: pg_partman: disable p_retention_schema parameter
 66ed3a5 [#23197]  YSQL: pg_partman: Disable Gist index creation
 3996f55 [YNP][PLAT-14664] make node register to provider idempotent
 Excluded: 6ec058d [PLAT-14668] - Move YSQL/YCQL configuration RBAC check to universe actions level and integrate RBAC for PG Compatibility
 Excluded: 18bb9b8 [#23034] YSQL: Add Support for OIDC IDP URL (jwt_jwks_url) to fetch and refresh JKWS
 68cb1d2 [PLAT-14675][YNP] Fix the formatting for preflight checks
 db445ce [PLAT-14471][PLAT-14576] Added configurable deadline, keepAlive and unavailable retries to Ybc Java client
 Excluded: 47da28c [#23192] YSQL: Simplify/cleanup code in PgDml/PgSelect/PgSelectIndex etc
 Excluded: 1bc7a50 fix : gflag group toggle issue

Test Plan: Jenkins: rebase: pg15-cherrypicks

Reviewers: jason, tfoucher

Tags: #jenkins-ready

Differential Revision: https://phorge.dev.yugabyte.com/D36624
arybochkin added a commit that referenced this issue Jul 17, 2024
@arybochkin
[BACKPORT pg15-cherrypicks][#22937] docdb: Backward scans: make pggat…
224372f 
…e be aware of fast backward scan capability

Summary:
This is a first part of CBO-related changes for backward scans, which is not directly related to
the cost calculation. But the aim of the change is to make pggate be aware of fast backward scan
capability to be able to correctly identify the cost of backward scans in the CBO.
Refer to #22370 for the details.

For this purpose the flag is moved to common_flags.cc and YBCPgGFlagsAccessor is updated
to be able to get the value of the flag. Additionally the flag is mark as `NON_RUNTIME` to prevent
any confusion from a user side as PG flags does not support runtime change. Such change is
acceptable as it is not expected the flag change in future.

**Upgrade/Rollback safety:**
In the worst case, when the node is not yet upgraded, the planner/optimizer will not be aware of
the fast backward scan capability and as a result would be choosing a different execution path
without fast backward scan optimization avoiding possible performance gain. On the other hand,
if the planner/optimizer already aware of the capability (and it's on), but remote yb-tserver is
not yet upgraded, then it may result in a slower query but with no correctness impact.

**Note.** Merge conflicts.
Original conflicts have been resolved by 259a71e

New conflicts are due to git merge was not able to handle a new field at the end of a structure, which was
last updated by YB pg15 fedbdac:
- ybc_pg_typedefs.h:397
```
  const bool*     TEST_ysql_hide_catalog_version_increment_log;
<<<<<<< HEAD
  const bool*     TEST_generate_ybrowid_sequentially;
=======
  const bool*     ysql_use_fast_backward_scan;
>>>>>>> 1773ae2 ([#22937] docdb: Backward scans: make pggate be aware of fast backward scan capability)
} YBCPgGFlagsAccessor;
```

- ybc_pggate.cc:1927
```
          &FLAGS_TEST_ysql_hide_catalog_version_increment_log,
<<<<<<< HEAD
      .TEST_generate_ybrowid_sequentially =
          &FLAGS_TEST_generate_ybrowid_sequentially,
=======
      .ysql_use_fast_backward_scan = &FLAGS_use_fast_backward_scan,
>>>>>>> 1773ae2 ([#22937] docdb: Backward scans: make pggate be aware of fast backward scan capability)
  };
```

Resolved them manually.

Jira: DB-11853

Original commit: 1773ae2 / D36245

Test Plan: Jenkins

Reviewers: jason, tfoucher

Reviewed By: jason

Subscribers: yql, ybase, hsunder

Tags: #jenkins-ready

Differential Revision: https://phorge.dev.yugabyte.com/D36653
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@arybochkin arybochkin

Labels
area/docdb YugabyteDB core features kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@yugabyte-ci @arybochkin