Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

expression: return error when doing ResolveIndices #8929

Merged
merged 11 commits into from
Jan 15, 2019

Conversation

winoros
Copy link
Member

@winoros winoros commented Jan 3, 2019

What problem does this PR solve?

Originally, there're some cases that ResolveIndices find out that there're some columns not resolved although the SQL can be executed successfully.
This pr fixes it and let ResolveIndices return error.

What is changed and how it works?

Fix the following cases:

  • Projection's columnPruning doesn't prune the _rowid column correctly
  • When it's a double read with keep order case. There'll be an extra _rowid column returned and changed the schema of operators above the DataSource operator.
  • When push agg down across union, union's schema is not set correctly.

And changed the ResolveIndices method let it return an error.

Check List

Tests

  • Unit test
  • Integration test

Code changes

  • Has exported function/method change
  • Has interface methods change

Side effects

  • Increased code complexity

Related changes

None


This change is Reviewable

@winoros
Copy link
Member Author

winoros commented Jan 3, 2019

/run-all-tests

@winoros
Copy link
Member Author

winoros commented Jan 7, 2019

/run-all-tests

@winoros
Copy link
Member Author

winoros commented Jan 7, 2019

/run-all-tests

@winoros
Copy link
Member Author

winoros commented Jan 8, 2019

/run-all-tests

@winoros
Copy link
Member Author

winoros commented Jan 8, 2019

/run-all-tests

@winoros winoros changed the title [WIP]expression: return error when doing ResolveIndices expression: return error when doing ResolveIndices Jan 8, 2019
col.Index = schema.ColumnIndex(col)
if col.Index == -1 {
log.Errorf("Can't find column %s in schema %s", col, schema)
return errors.Trace(errors.Errorf("Can't find column %s in schema %s", col, schema))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove errors.Trace?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Errorf won't add StackTree automatically.
And error stack is helpful for this kind of error.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

errors.Trace() will be removed in the near future. It's better to generate an error with stack and remove the errors.Trace() function call.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

errors.ErrorfWithStack(format string, args ...interface{})?
pingcap/errors have a method named WithStack(err) which i think it's the same thing with errors.Trace

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After reading the implementation of errors.Errorf(), I think the generated error already has a stack:

103 // Errorf formats according to a format specifier and returns the string
104 // as a value that satisfies error.
105 // Errorf also records the stack trace at the point it was called.
106 func Errorf(format string, args ...interface{}) error {
107     return &fundamental{
108         msg:   fmt.Sprintf(format, args...),
109         stack: callers(),
110     }
111 }

I think you can directly call errors.Errorf() to generate an error with stack trace.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, you're right. Changed

planner/core/rule_column_pruning.go Show resolved Hide resolved
planner/core/task.go Outdated Show resolved Hide resolved
col.Index = schema.ColumnIndex(col)
if col.Index == -1 {
log.Errorf("Can't find column %s in schema %s", col, schema)
return errors.Trace(errors.Errorf("Can't find column %s in schema %s", col, schema))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

errors.Trace() will be removed in the near future. It's better to generate an error with stack and remove the errors.Trace() function call.

planner/core/resolve_indices.go Outdated Show resolved Hide resolved
@codecov-io
Copy link

codecov-io commented Jan 10, 2019

Codecov Report

Merging #8929 into master will decrease coverage by 0.07%.
The diff coverage is 49.45%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #8929      +/-   ##
==========================================
- Coverage   67.25%   67.17%   -0.08%     
==========================================
  Files         371      371              
  Lines       76223    76393     +170     
==========================================
+ Hits        51261    51318      +57     
- Misses      20424    20490      +66     
- Partials     4538     4585      +47
Impacted Files Coverage Δ
expression/expression.go 85.49% <ø> (ø) ⬆️
executor/builder.go 82.61% <ø> (-0.04%) ⬇️
planner/core/plan.go 95% <ø> (ø) ⬆️
planner/core/optimizer.go 61.53% <100%> (ø) ⬆️
planner/core/find_best_task.go 85.83% <100%> (+0.04%) ⬆️
planner/core/rule_aggregation_push_down.go 84.68% <100%> (+0.06%) ⬆️
expression/constant.go 49.44% <100%> (+0.28%) ⬆️
planner/core/logical_plan_builder.go 74.49% <100%> (ø) ⬆️
planner/core/task.go 95.93% <100%> (+0.09%) ⬆️
planner/core/point_get_plan.go 79.27% <20%> (-0.8%) ⬇️
... and 10 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e6a0eb9...f6140ca. Read the comment docs.

Copy link
Member

@zz-jason zz-jason left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zz-jason zz-jason added the status/LGT1 Indicates that a PR has LGTM 1. label Jan 14, 2019
Copy link
Contributor

@alivxxx alivxxx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@alivxxx alivxxx added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jan 15, 2019
@winoros winoros merged commit dca815c into pingcap:master Jan 15, 2019
@winoros winoros deleted the resolve_indices branch January 15, 2019 06:34
winoros added a commit to winoros/tidb that referenced this pull request Aug 22, 2019
sre-bot pushed a commit that referenced this pull request Aug 22, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/expression status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants