Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plan, expression: support is null in hash partition pruning (#17281) #17308

Merged
merged 8 commits into from
Jun 28, 2020

Conversation

sre-bot
Copy link
Contributor

@sre-bot sre-bot commented May 20, 2020

cherry-pick #17281 to release-3.0


What problem does this PR solve?

Problem Summary:

SQL like select * from t where a is null will produce a full table scan plan in hash partition. For example:

create table t (a int, b int) partition by hash(a + b) partitions 10;
explain select * from t where a is null and b = 2;

This SQL will produce following result:

MySQL [test]> explain select * from t where a is null and b = 2;
+------------------------------+----------+-----------+-----------------------+-----------------------------------+
| id                           | estRows  | task      | access object         | operator info                     |
+------------------------------+----------+-----------+-----------------------+-----------------------------------+
| Union_16                     | 0.10     | root      |                       |                                   |
| ├─TableReader_19             | 0.01     | root      |                       | data:Selection_18                 |
| │ └─Selection_18             | 0.01     | cop[tikv] |                       | eq(test.t.b, 2), isnull(test.t.a) |
| │   └─TableFullScan_17       | 10000.00 | cop[tikv] | table:t, partition:p0 | keep order:false, stats:pseudo    |
| ├─TableReader_22             | 0.01     | root      |                       | data:Selection_21                 |
| │ └─Selection_21             | 0.01     | cop[tikv] |                       | eq(test.t.b, 2), isnull(test.t.a) |
| │   └─TableFullScan_20       | 10000.00 | cop[tikv] | table:t, partition:p1 | keep order:false, stats:pseudo    |
| ├─TableReader_25             | 0.01     | root      |                       | data:Selection_24                 |
| │ └─Selection_24             | 0.01     | cop[tikv] |                       | eq(test.t.b, 2), isnull(test.t.a) |
| │   └─TableFullScan_23       | 10000.00 | cop[tikv] | table:t, partition:p2 | keep order:false, stats:pseudo    |
| ├─TableReader_28             | 0.01     | root      |                       | data:Selection_27                 |
| │ └─Selection_27             | 0.01     | cop[tikv] |                       | eq(test.t.b, 2), isnull(test.t.a) |
| │   └─TableFullScan_26       | 10000.00 | cop[tikv] | table:t, partition:p3 | keep order:false, stats:pseudo    |
| ├─TableReader_31             | 0.01     | root      |                       | data:Selection_30                 |
| │ └─Selection_30             | 0.01     | cop[tikv] |                       | eq(test.t.b, 2), isnull(test.t.a) |
| │   └─TableFullScan_29       | 10000.00 | cop[tikv] | table:t, partition:p4 | keep order:false, stats:pseudo    |
| ├─TableReader_34             | 0.01     | root      |                       | data:Selection_33                 |
| │ └─Selection_33             | 0.01     | cop[tikv] |                       | eq(test.t.b, 2), isnull(test.t.a) |
| │   └─TableFullScan_32       | 10000.00 | cop[tikv] | table:t, partition:p5 | keep order:false, stats:pseudo    |
| ├─TableReader_37             | 0.01     | root      |                       | data:Selection_36                 |
| │ └─Selection_36             | 0.01     | cop[tikv] |                       | eq(test.t.b, 2), isnull(test.t.a) |
| │   └─TableFullScan_35       | 10000.00 | cop[tikv] | table:t, partition:p6 | keep order:false, stats:pseudo    |
| ├─TableReader_40             | 0.01     | root      |                       | data:Selection_39                 |
| │ └─Selection_39             | 0.01     | cop[tikv] |                       | eq(test.t.b, 2), isnull(test.t.a) |
| │   └─TableFullScan_38       | 10000.00 | cop[tikv] | table:t, partition:p7 | keep order:false, stats:pseudo    |
| ├─TableReader_43             | 0.01     | root      |                       | data:Selection_42                 |
| │ └─Selection_42             | 0.01     | cop[tikv] |                       | eq(test.t.b, 2), isnull(test.t.a) |
| │   └─TableFullScan_41       | 10000.00 | cop[tikv] | table:t, partition:p8 | keep order:false, stats:pseudo    |
| └─TableReader_46             | 0.01     | root      |                       | data:Selection_45                 |
|   └─Selection_45             | 0.01     | cop[tikv] |                       | eq(test.t.b, 2), isnull(test.t.a) |
|     └─TableFullScan_44       | 10000.00 | cop[tikv] | table:t, partition:p9 | keep order:false, stats:pseudo    |
+------------------------------+----------+-----------+-----------------------+-----------------------------------+

What is changed and how it works?

Proposal: xxx

What's Changed:

During partition pruning, if a column is equal to NULL, all calculation relating to it will produce null result and set a NULL flag.

How it Works:

If pruning result with a NULL flag, it will be point to p0.

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test

Side effects

  • No

Release note

  • planner: support is null filter condition in hash partition pruning

Signed-off-by: sre-bot <sre-bot@pingcap.com>
@sre-bot
Copy link
Contributor Author

sre-bot commented May 20, 2020

/run-all-tests

@tiancaiamao
Copy link
Contributor

LGTM

@tiancaiamao
Copy link
Contributor

/run-all-tests

@bb7133 bb7133 modified the milestones: v3.0.15, 3.0.16, v3.0.16 Jun 6, 2020
@imtbkcat
Copy link

/rebuild

@imtbkcat
Copy link

PTAL @XuHuaiyu

Copy link
Contributor

@qw4990 qw4990 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot
Copy link
Contributor

@qw4990,Thanks for you review.

@qw4990 qw4990 added the status/LGT2 Indicates that a PR has LGTM 2. label Jun 28, 2020
Copy link
Member

@zz-jason zz-jason left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot removed the status/LGT2 Indicates that a PR has LGTM 2. label Jun 28, 2020
@ti-srebot
Copy link
Contributor

@zz-jason,Thanks for you review.

@zz-jason zz-jason added status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. labels Jun 28, 2020
@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot ti-srebot merged commit 6609b1f into pingcap:release-3.0 Jun 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/expression sig/planner SIG: Planner status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/bugfix This PR fixes a bug. type/3.0-cherry-pick
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants