Remove failed task distributor #687
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## perf_coll_fw_enhance #687 +/- ##
========================================================
- Coverage 70.96% 70.92% -0.05%
========================================================
Files 161 160 -1
Lines 15290 15240 -50
Branches 1872 1868 -4
========================================================
- Hits 10851 10809 -42
+ Misses 3821 3814 -7
+ Partials 618 617 -1
|
ThisIsClark
force-pushed
the
remove_failed_job_distributor
branch
from
d84cc70
to
6a4cc41
Compare
ThisIsClark
force-pushed
the
remove_failed_job_distributor
branch
from
6a4cc41
to
5a889c2
Compare
|
LGTM |
NajmudheenCT
reviewed
Sep 7, 2021
| @@ -108,3 +108,6 @@ def _handle_task_failure(self, start_time, end_time): | |||
| FailedTask.retry_count.name: 0, | |||
| FailedTask.executor.name: self.executor} | |||
| db.failed_task_create(self.ctx, failed_task) | |||
| self.metric_task_rpcapi.assign_failed_job(self.ctx, | |||
There was a problem hiding this comment.
This message will help to assign failed job in same executor.. But with removal failed_job distributor scanning there no way to assign failed_job in case of a node down/restart .
There was a problem hiding this comment.
Ok, so we should add another mechanism to recover the failed job which belongs to this executor
kumarashit
added a commit
that referenced
this pull request
Sep 14, 2021
* Make job scheduler local to task process (#674) * Make job scheduler local to task process * Notify distributor when a new task added (#678) * Remove db-scan for new task creation (#680) * Use consistent hash to manage the topic (#681) * Remove the periodically call from task distributor (#686) * Start one historic collection immediate when a job is rescheduled (#685) * Start one historic collection immediate when a job is rescheduled * Remove failed task distributor (#687) * Improving Failed job handling and telemetry job removal (#689) Co-authored-by: ThisIsClark <liuyuchibubao@gmail.com> Co-authored-by: Ashit Kumar <akopensrc@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Remove failed task distributor
Which issue this PR fixes (optional, in
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close that issue when PR gets merged): fixes #Special notes for your reviewer:
Release note: