planner, executor: fix index merge partial table scan schema

[tangenta]

What problem does this PR solve?

Issue Number: close #23919

Problem Summary:

create table t (a int, b int, index(a), index(b)) partition by hash (a) partitions 2;
insert into t values (1, 5);
select /*+ use_index_merge( t ) */ * from t where a in (3) or b in (5) order by a;

ERROR 1105 (HY000): runtime error: index out of range [2] with length 2

The explain result:

+--------------------------------------+---------+-----------+-----------------------------------+---------------------------------------------+
| id                                   | estRows | task      | access object                     | operator info                               |
+--------------------------------------+---------+-----------+-----------------------------------+---------------------------------------------+
| Sort_9                               | 39.98   | root      |                                   | test.t.a                                    |
| └─PartitionUnion_12                  | 39.98   | root      |                                   |                                             |
|   ├─IndexMerge_16                    | 19.99   | root      |                                   |                                             |
|   │ ├─IndexRangeScan_13(Build)       | 10.00   | cop[tikv] | table:t, partition:p0, index:a(a) | range:[3,3], keep order:false, stats:pseudo |
|   │ ├─IndexRangeScan_14(Build)       | 10.00   | cop[tikv] | table:t, partition:p0, index:b(b) | range:[5,5], keep order:false, stats:pseudo |
|   │ └─TableRowIDScan_15(Probe)       | 19.99   | cop[tikv] | table:t, partition:p0             | keep order:false, stats:pseudo              |
|   └─IndexMerge_20                    | 19.99   | root      |                                   |                                             |
|     ├─IndexRangeScan_17(Build)       | 10.00   | cop[tikv] | table:t, partition:p1, index:a(a) | range:[3,3], keep order:false, stats:pseudo |
|     ├─IndexRangeScan_18(Build)       | 10.00   | cop[tikv] | table:t, partition:p1, index:b(b) | range:[5,5], keep order:false, stats:pseudo |
|     └─TableRowIDScan_19(Probe)       | 19.99   | cop[tikv] | table:t, partition:p1             | keep order:false, stats:pseudo              |
+--------------------------------------+---------+-----------+-----------------------------------+---------------------------------------------+

SortExec.Next() only requires 2 columns(a and b), but the row fetched from the underlying UnionExec has 3(a, b and _tidb_rowid). This cause Chunk.AppendPartialRow panic because the slice out-of-range error.

The request chunk for UnionExec.Next should have only 2 columns, but req.SwapColumns(result.chk) change it into a 3-columns chunk. This chunk comes from IndexMergeReaderExecutor, which has a 3-columns schema.

IndexMergeReaderExecutor inherits the schema from PhysicalIndexMergeReader, which inherits from the PhysicalTableScan built by buildIndexMergeTableScan:

tidb/planner/core/find_best_task.go

Lines 903 to 914 in 008b91b

	if ts.HandleCols == nil {
	handleCol := ds.getPKIsHandleCol()
	if handleCol == nil {
	handleCol, _ = ts.appendExtraHandleCol(ds)
	}
	ts.HandleCols = NewIntHandleCols(handleCol)
	}
	var err error
	ts.HandleCols, err = ts.HandleCols.ResolveIndices(ts.schema)
	if err != nil {
	return nil, 0, err
	}

It appends the _tidb_rowid to the schema of DataSource to help HandleCols resolve its indices.

The HandleCols is used to fetch handles in both TableRangeScan and IndexRangeScan.

• For the TableRangeScan, the current schema is from DataSource plus additional common handle columns. This is unresonable because only the handle columns are needed.
• For the IndexRangeScan, the current schema is always indexed columns plus full common handle columns. ResolveIndices is useless in IndexRangeScan because the expression.Column.Index is never used.

What is changed and how it works?

How it Works:

This PR makes the schema of IndexMerge consistent with the upper operator, by avoid changing DataSource schema directly. It also fixes the issue in #23933 (review).

What is changed:

• Change the schema of TableRangeScan(as a child of IndexMerge operator) to the one that only contains handle columns.
• Resolve the HandleCols in IndexMergeReaderExecutor against the schema that only contains the handle columns.
• Remove unnecessary projection.
• Add test for sorting index merge result in partition union reports 'index out of range' #23919 and clustered index/_tidb_rowid/PkIsHandle for index merge.

Related changes

NA

Check List

Tests

• Unit test
• Integration test

Side effects

NA

Release note

• Fix an issue that sorting on index-merge results in partition union reports 'index out of range'.

[tangenta]

/hold

[tangenta]

/unhold

[eurekaka]

/lgtm

[ti-chi-bot]

@eurekaka: /lgtm is only allowed for the reviewers in list.

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[eurekaka]

/lgtm

[eurekaka]

/lgtm

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has been approved by:

• eurekaka
• lzmhhh123

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by writing /lgtm in a comment.
Reviewer can cancel approval by writing /lgtm cancel in a comment.

[eurekaka]

/merge

[ti-chi-bot]

This pull request has been accepted and is ready to merge.

Commit hash: 42e3493

[ti-srebot]

cherry pick to release-5.0 in PR #24155