-
Notifications
You must be signed in to change notification settings - Fork 378
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
limit task retry when system error #3340
limit task retry when system error #3340
Conversation
Codecov Report
@@ Coverage Diff @@
## master #3340 +/- ##
=========================================
Coverage 17.85% 17.85%
=========================================
Files 1377 1386 +9
Lines 142888 144326 +1438
=========================================
+ Hits 25511 25773 +262
- Misses 114700 115875 +1175
- Partials 2677 2678 +1
|
26f9358
to
4d61ffd
Compare
apistructs/pipeline_task.go
Outdated
@@ -23,7 +23,9 @@ import ( | |||
|
|||
const ( | |||
// TerminusDefineTag add this tag env to container for collecting logs | |||
TerminusDefineTag = "TERMINUS_DEFINE_TAG" | |||
TerminusDefineTag = "TERMINUS_DEFINE_TAG" | |||
PipelineTaskMaxRetryLimit = 100 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How much time will the normal result of 100 retry times cost?
Please compare to 24hours.
@@ -96,6 +96,10 @@ func reconcileTask(tr *taskrun.TaskRun) error { | |||
rlog.TErrorf(tr.P.ID, tr.Task.ID, "failed to handle taskOp: %s, user abnormalErr: %v, don't need retry", taskOp.Op(), abnormalErr) | |||
return abnormalErr | |||
} | |||
if tr.Task.Inspect.IsErrorsExceed() { | |||
rlog.TErrorf(tr.P.ID, tr.Task.ID, "failed to handle taskOp: %s, errors exceed limit, stop retry", taskOp.Op()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
log add concrete limit number.
4d61ffd
to
942443f
Compare
/approve |
What type of this PR
Add one of the following kinds:
/kind bugfix
What this PR does / why we need it:
limit task retry when system error
Which issue(s) this PR fixes:
Specified Reviewers:
/assign @your-reviewer
ChangeLog
Need cherry-pick to release versions?
Add comment like
/cherry-pick release/1.0
when this PR is merged.