This repository has been archived by the owner on Nov 24, 2023. It is now read-only.
loader/syncer: filter context cancel error while executing sqls #355
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lichunzhu
added
priority/normal
lichunzhu
added
status/WIP
Codecov Report
@@ Coverage Diff @@
## master #355 +/- ##
================================================
+ Coverage 56.5969% 57.6986% +1.1016%
================================================
Files 163 162 -1
Lines 16697 16373 -324
================================================
- Hits 9450 9447 -3
+ Misses 6324 6001 -323
- Partials 923 925 +2 |
|
/run-all-tests tidb=release-3.0 |
lichunzhu
added
status/PTAL
csuzhangxc
reviewed
Nov 22, 2019
loader/loader.go
Outdated
| @@ -174,7 +174,11 @@ func (w *Worker) run(ctx context.Context, fileJobQueue chan *fileJob, runFatalCh | |||
| if err != nil { | |||
| // expect pause rather than exit | |||
| err = terror.WithScope(terror.Annotatef(err, "file %s", job.file), terror.ScopeDownstream) | |||
| runFatalChan <- unit.NewProcessError(pb.ErrorType_ExecSQL, err) | |||
| if utils.IsContextCanceledError(err) { | |||
| runFatalChan <- nil | |||
There was a problem hiding this comment.
If we don't deal with runFatalChan, the user will still get cancel error from dmctl. Shall we revise unit.NewProcessError to filter that instead of this way?
There was a problem hiding this comment.
if err is context cancel error, don't need to send nil to runFatalChan
…chunzhu/dm into czli/filterContextCancelError
|
/run-all-tests tidb=release-3.0 |
csuzhangxc
reviewed
Nov 25, 2019
pkg/log/log.go
Outdated
| @@ -130,7 +132,7 @@ func SetLevel(level zapcore.Level) zapcore.Level { | |||
| // just repeats known information. You should almost always use `ShortError` | |||
| // instead of `zap.Error`, unless the error is no longer propagated upwards. | |||
| func ShortError(err error) zap.Field { | |||
| if err == nil { | |||
| if err == nil || errors.Cause(err) == context.Canceled { | |||
There was a problem hiding this comment.
There was a problem hiding this comment.
Do you mean put err == nil || errors.Cause(err) == context.Canceled... into a new function?
@WangXiangUSTC
There was a problem hiding this comment.
how about adding a new function to judge whether an error can be filter
func FilterError(err error) error {
if errors.Cause(err) == context.Canceled {
return nil
}
return err
}
then we can use ShortError(FilterError(err))
There was a problem hiding this comment.
approve the method from @WangXiangUSTC, but maybe a meaningful name needed?
There was a problem hiding this comment.
How about FilterCancelError?
There was a problem hiding this comment.
@csuzhangxc
which one is better:
- return
error. the usage would belog.ShortError(log.FilterCancelError(err)) - return
zap.Field. the usage would belog.FilterCancelError(err)
…erContextCancelError
…chunzhu/dm into czli/filterContextCancelError
pkg/log/log.go
Outdated
| @@ -136,6 +138,14 @@ func ShortError(err error) zap.Field { | |||
| return zap.String("error", err.Error()) | |||
| } | |||
|
|
|||
| // FilterCancelError will skip context.Canceled error | |||
| func FilterCancelError(err error) error { | |||
| if errors.Cause(err) == context.Canceled { | |||
There was a problem hiding this comment.
how about use IsContextCanceledError?
|
/run-all-tests tidb=release-3.0 |
|
@csuzhangxc Revised, PTAL |
csuzhangxc
added
status/LGT1
|
/run-all-tests tidb=release-3.0 |
|
cherry pick to release-1.0 failed |
This was referenced Nov 28, 2019
WangXiangUSTC
added
status/LGT2
lichunzhu
added a commit
that referenced
this pull request
Nov 29, 2019
csuzhangxc
pushed a commit
that referenced
this pull request
Nov 29, 2019
csuzhangxc
added
already-cherry-pick-1.0
lichunzhu
added a commit
to lichunzhu/dm
that referenced
this pull request
Apr 6, 2020
…cap#355) * add ErrorFilterContextCanceled to skip error log of context.Canceled
lichunzhu
added a commit
to lichunzhu/dm
that referenced
this pull request
Apr 6, 2020
* fix situations when there is extra msg in error.Error * add deal for zap.Error and add UT
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
When an error occurs and the context in loader/syncer is canceled, the normally running sql will be canceled and return a
context.Cancelederror, which will confuse the users.What is changed and how it works?
Filter all the
context.Canceledin loader/syncer. Don't return error and don't print log.Check List
Tests
Related changes