Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

planner: fix should not use point get plan #21124

Merged
merged 13 commits into from
Nov 24, 2020

Conversation

blacktear23
Copy link
Contributor

@blacktear23 blacktear23 commented Nov 18, 2020

What problem does this PR solve?

Issue Number: close #21123 , close #21204

Problem Summary:

In MySQL string convert to numbers it will use number prefix to convert to number, for example:

'asdf' => 0
'123asdf' => 123

So if we create a table like create table t (id varchar(10) primary key), then insert ('asdf'), ('1asdf') into this table. When we execute a query like:

select * from t where id = 0;

the plan should be convert column id into int and compare with number 0, so we should get 'asdf' as the result. But in TiDB it will convert number 0 to string '0' and use point get plan to fetch id = '0' rows, so we will get an empty set result.

What is changed and how it works?

What's Changed:

Add type check for TryFastPlan, if column type is string like and in or = value is number just return empty plan.

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Integration test

Side effects

  • Performance regression
    • Consumes more CPU

Release note

  • planner: fix wrong point get plan generation in fast plan code path

@blacktear23 blacktear23 requested a review from a team as a code owner November 18, 2020 15:30
@blacktear23 blacktear23 requested review from winoros and removed request for a team November 18, 2020 15:30
@ichn-hu ichn-hu mentioned this pull request Nov 18, 2020
planner/core/point_get_plan.go Outdated Show resolved Hide resolved
dKind := d.Kind()
switch col.FieldType.EvalType() {
case ptypes.ETString:
switch dKind {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use the white list?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now I just find String type column with numeric constant will cause the bug. So I just use black list style.

tk.MustExec("create table t3(id varchar(20), d varchar(20), unique(id, d))")
tk.MustExec("create table t4(id int, d varchar(20), c varchar(20), unique(id, d))")

tk.MustQuery("explain select * from t1 where id = 0").Check(testkit.Rows(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use the test data to record the plan test.

case ptypes.ETString:
switch kind {
case types.KindInt64, types.KindUint64,
types.KindFloat32, types.KindFloat64, types.KindMysqlDecimal:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should bit be contained here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will do bit type test on mysql later.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, please also test year and other int like types.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

datetime serise is tested, no problem now. do we need test case for this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's much better. More test cases of different types could help us avoid this problem in the future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lzmhhh123 bit and Hex binary literal tested same as MySQL, so just char column with numeric literal need to be reject from try point get plan.

@blacktear23
Copy link
Contributor Author

@lzmhhh123 datetime series literal equals with char column has some corner case. But with numeric literal and bit or hex literal is perform same as MySQL.

@lzmhhh123
Copy link
Contributor

@lzmhhh123 datetime series literal equals with char column has some corner case. But with numeric literal and bit or hex literal is perform same as MySQL.

The corner case between datetime and char could be fixed in another PR.

Copy link
Contributor

@lzmhhh123 lzmhhh123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Nov 20, 2020
@blacktear23
Copy link
Contributor Author

@lzmhhh123 PTAL #21167

Copy link
Member

@wjhuang2016 wjhuang2016 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot removed the status/LGT1 Indicates that a PR has LGTM 1. label Nov 20, 2020
ti-srebot
ti-srebot previously approved these changes Nov 20, 2020
@ti-srebot ti-srebot added the status/LGT2 Indicates that a PR has LGTM 2. label Nov 20, 2020
@crazycs520
Copy link
Contributor

@blacktear23 I found a related bug too, Could you take a look? #21204

@crazycs520
Copy link
Contributor

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Nov 23, 2020
@ti-srebot
Copy link
Contributor

Your auto merge job has been accepted, waiting for:

  • 21117

@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot
Copy link
Contributor

@blacktear23 merge failed.

@wjhuang2016
Copy link
Member

/merge

@ti-srebot
Copy link
Contributor

Your auto merge job has been accepted, waiting for:

  • 21201

@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot
Copy link
Contributor

@blacktear23 merge failed.

@blacktear23
Copy link
Contributor Author

@blacktear23 I found a related bug too, Could you take a look? #21204

Yes

@blacktear23
Copy link
Contributor Author

@crazycs520 PTAL

@bb7133
Copy link
Member

bb7133 commented Nov 23, 2020

/run-all-tests

@wjhuang2016
Copy link
Member

/merge

@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot ti-srebot added status/LGT3 The PR has already had 3 LGTM. and removed status/LGT2 Indicates that a PR has LGTM 2. labels Nov 24, 2020
Copy link
Contributor

@crazycs520 crazycs520 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@crazycs520
Copy link
Contributor

/merge

@ti-srebot
Copy link
Contributor

Your auto merge job has been accepted, waiting for:

  • 21062
  • 21182

@ti-srebot
Copy link
Contributor

@blacktear23 merge failed.

@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot ti-srebot merged commit 8808a65 into pingcap:master Nov 24, 2020
@blacktear23
Copy link
Contributor Author

@bb7133 do we need cherry-pick?

@lzmhhh123
Copy link
Contributor

/run-cherry-picker

ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Nov 24, 2020
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot
Copy link
Contributor

cherry pick to release-4.0 in PR #21244

ti-srebot added a commit that referenced this pull request Nov 25, 2020
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status/can-merge Indicates a PR has been approved by a committer. status/LGT3 The PR has already had 3 LGTM.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

planner: Point get plan will cause wrong result set planner: Point get plan will cause wrong result set
6 participants