Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

executor: support index merge on cluster index #18699

Merged

Conversation

lzmhhh123
Copy link
Contributor

What problem does this PR solve?

Problem Summary: as title says.

What is changed and how it works?

What's Changed: support index merge on cluster index

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Integration test

Side effects

  • Performance regression
    • Consumes more CPU
    • Consumes more MEM

Release note

  • none

@lzmhhh123 lzmhhh123 requested review from a team as code owners July 21, 2020 03:30
@lzmhhh123 lzmhhh123 requested review from wshwsh12, XuHuaiyu and coocood and removed request for a team July 21, 2020 03:30
@lzmhhh123 lzmhhh123 added the sig/execution SIG execution label Jul 21, 2020
@@ -2681,6 +2681,10 @@ func (b *executorBuilder) buildIndexMergeReader(v *plannercore.PhysicalIndexMerg
sctx.IndexNames = append(sctx.IndexNames, is.Table.Name.O+":"+is.Index.Name.O)
} else {
ret.ranges = append(ret.ranges, v.PartialPlans[i][0].(*plannercore.PhysicalTableScan).Ranges)
if ret.table.Meta().IsCommonHandle {
tblInfo := ret.table.Meta()
sctx.IndexNames = append(sctx.IndexNames, tblInfo.Name.O+":"+tables.FindPrimaryIndex(tblInfo).Name.O)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we simply use PRIMARY?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It keeps the same name as the unclustered primary key.

@@ -359,8 +383,11 @@ func (w *partialTableWorker) extractTaskHandles(ctx context.Context, chk *chunk.
return handles, retChk, nil
}
for i := 0; i < chk.NumRows(); i++ {
h := kv.IntHandle(chk.GetRow(i).GetInt64(handleOffset))
handles = append(handles, h)
handle, err := handleCols.BuildHandleByOffsets(chk.GetRow(i), handleOffset)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The handleCols has resolved column index, can we call BuildHandle without handleOffset?

handleOffset := chk.NumCols() - 1
var handleOffset []int
for i := 0; i < handleCols.NumCols(); i++ {
handleOffset = append(handleOffset, chk.NumCols()-handleCols.NumCols()+i)
Copy link
Member

@coocood coocood Jul 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the index row's handle columns always at the last, we can move this logic into HandleCols, and use a method BuildHandleFromIndexRow.
We don't need to build the handleOffsets.

@lzmhhh123 lzmhhh123 closed this Jul 21, 2020
@lzmhhh123 lzmhhh123 reopened this Jul 21, 2020
@lzmhhh123
Copy link
Contributor Author

/rebuild

@coocood
Copy link
Member

coocood commented Jul 21, 2020

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Jul 21, 2020
@coocood coocood requested a review from eurekaka July 24, 2020 04:38
@coocood
Copy link
Member

coocood commented Jul 24, 2020

@eurekaka PTAL

Copy link
Contributor

@eurekaka eurekaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot
Copy link
Contributor

@eurekaka,Thanks for your review. However, LGTM is restricted to Reviewers or higher roles.See the corresponding SIG page for more information. Related SIGs: execution(slack).

@coocood coocood added the sig/planner SIG: Planner label Jul 27, 2020
Copy link
Contributor

@eurekaka eurekaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot removed the status/LGT1 Indicates that a PR has LGTM 1. label Jul 27, 2020
@ti-srebot ti-srebot added the status/LGT2 Indicates that a PR has LGTM 2. label Jul 27, 2020
@coocood
Copy link
Member

coocood commented Jul 27, 2020

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Jul 27, 2020
@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot
Copy link
Contributor

@lzmhhh123 merge failed.

@codecov
Copy link

codecov bot commented Jul 27, 2020

Codecov Report

Merging #18699 into master will increase coverage by 0.5102%.
The diff coverage is 71.1206%.

@@               Coverage Diff                @@
##             master     #18699        +/-   ##
================================================
+ Coverage   79.2262%   79.7364%   +0.5102%     
================================================
  Files           542        544         +2     
  Lines        146189     149870      +3681     
================================================
+ Hits         115820     119501      +3681     
+ Misses        21034      21025         -9     
- Partials       9335       9344         +9     

@lzmhhh123
Copy link
Contributor Author

/merge

@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot ti-srebot merged commit 56fd348 into pingcap:master Jul 27, 2020
@lzmhhh123 lzmhhh123 deleted the dev/support_clustered_index_in_index_merge branch August 14, 2020 04:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/execution SIG execution sig/planner SIG: Planner status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants