Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix No Matching column bug #1162

Merged

Conversation

marsishandsome
Copy link
Collaborator

@marsishandsome marsishandsome commented Oct 21, 2019

What problem does this PR solve?

close #1161

CREATE TABLE `t_no_match_column` (
  `json` text DEFAULT NULL
)
select temp44.id from (
    SELECT get_json_object(t.json,  '$.id') as id from t_no_match_column t
) as temp44 order by temp44.id desc limit 1
No Matching column id from table t_no_match_column
com.pingcap.tikv.exception.TiExpressionException: No Matching column id from table t_no_match_column
	at com.pingcap.tikv.expression.ColumnRef.resolve(ColumnRef.java:71)
	at com.pingcap.tikv.expression.visitor.MetaResolver.visit(MetaResolver.java:52)
	at com.pingcap.tikv.expression.visitor.MetaResolver.visit(MetaResolver.java:24)
	at com.pingcap.tikv.expression.ColumnRef.accept(ColumnRef.java:141)
	at com.pingcap.tikv.expression.visitor.MetaResolver.lambda$resolve$0(MetaResolver.java:42)
	at java.lang.Iterable.forEach(Iterable.java:75)
	at com.pingcap.tikv.expression.visitor.MetaResolver.resolve(MetaResolver.java:42)
	at com.pingcap.tikv.meta.TiDAGRequest.resolve(TiDAGRequest.java:264)
	at org.apache.spark.sql.TiStrategy.toCoprocessorRDD(TiStrategy.scala:159)
	at org.apache.spark.sql.TiStrategy.pruneFilterProject(TiStrategy.scala:416)
	at org.apache.spark.sql.TiStrategy.pruneTopNFilterProject(TiStrategy.scala:312)
	at org.apache.spark.sql.TiStrategy.takeOrderedAndProject(TiStrategy.scala:351)
	at org.apache.spark.sql.TiStrategy.org$apache$spark$sql$TiStrategy$$doPlan(TiStrategy.scala:521)
	at org.apache.spark.sql.TiStrategy$$anonfun$apply$1.applyOrElse(TiStrategy.scala:146)
	at org.apache.spark.sql.TiStrategy$$anonfun$apply$1.applyOrElse(TiStrategy.scala:144)

What is changed and how it works?

This is a hotfix.
Add a config spark.tispark.plan.allow_order_project_limit_pushdown, default value = false.

will solve this problem in #1163

Check List

Tests

  • Unit test

Related changes

  • Need to cherry-pick to the release branch
  • Need to be included in the release note

@marsishandsome
Copy link
Collaborator Author

/run-all-tests

@marsishandsome marsishandsome added needs-cherry-pick-2.2 needs-cherry-pick-master PR which needs to be cherry-picked to master labels Oct 21, 2019
@marsishandsome
Copy link
Collaborator Author

/run-all-tests

birdstorm
birdstorm previously approved these changes Oct 21, 2019
Copy link
Contributor

@birdstorm birdstorm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, should clean up some code.

@marsishandsome
Copy link
Collaborator Author

/run-all-tests

@marsishandsome
Copy link
Collaborator Author

/build

@marsishandsome
Copy link
Collaborator Author

/run-all-tests

@marsishandsome
Copy link
Collaborator Author

/build

@marsishandsome
Copy link
Collaborator Author

/run-all-tests

@marsishandsome
Copy link
Collaborator Author

/run-all-tests

@marsishandsome
Copy link
Collaborator Author

/run-all-tests

1 similar comment
@zyxbest
Copy link

zyxbest commented Oct 22, 2019

/run-all-tests

@zyxbest
Copy link

zyxbest commented Oct 22, 2019

/build

1 similar comment
@sykp241095
Copy link
Member

/build

@sykp241095
Copy link
Member

/run-all-tests

1 similar comment
@marsishandsome
Copy link
Collaborator Author

/run-all-tests

@codecov-io
Copy link

codecov-io commented Oct 22, 2019

Codecov Report

Merging #1162 into release-2.1 will decrease coverage by 0.09%.
The diff coverage is n/a.

Impacted file tree graph

@@               Coverage Diff                @@
##             release-2.1    #1162     +/-   ##
================================================
- Coverage          53.36%   53.26%   -0.1%     
+ Complexity          1122     1119      -3     
================================================
  Files                153      153             
  Lines               7021     7021             
  Branches             819      819             
================================================
- Hits                3747     3740      -7     
- Misses              2902     2911      +9     
+ Partials             372      370      -2
Impacted Files Coverage Δ Complexity Δ
...lient/src/main/java/com/pingcap/tikv/PDClient.java 67.25% <0%> (-4.1%) 29% <0%> (-3%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6daab4e...5b58a2d. Read the comment docs.

@marsishandsome
Copy link
Collaborator Author

/run-all-tests

@marsishandsome
Copy link
Collaborator Author

/run-all-tests

Copy link
Contributor

@birdstorm birdstorm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@marsishandsome
Copy link
Collaborator Author

/run-all-tests tikv=v3.0.2 tidb=v3.0.2 pd=v3.0.2

@marsishandsome marsishandsome merged commit c8e9b34 into pingcap:release-2.1 Oct 23, 2019
marsishandsome added a commit to marsishandsome/tispark that referenced this pull request Oct 23, 2019
marsishandsome added a commit to marsishandsome/tispark that referenced this pull request Oct 23, 2019
wfxxh pushed a commit to wanfangdata/tispark that referenced this pull request Jun 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-cherry-pick-master PR which needs to be cherry-picked to master
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants