Skip to content

Commit

Permalink
Only report jobs done once their state has been reported (#899)
Browse files Browse the repository at this point in the history
Normally the state gets reported instantly so this is already true 99% of the time.

However if reporting the state goes wrong, we shouldn't report the job as done
 - Otherwise the server will tell the executor to kill the pod when it tries to maintain the lease

In all other places we make sure the JobEvent has been reported first before reporting done, so we should do that here too

This will only really impact edge cases and most of the time this will already be true
  • Loading branch information
JamesMurkin authored Mar 24, 2022
1 parent 4adb435 commit 5dfb12a
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion internal/executor/service/util.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ func jobShouldBeRenewed(job *job.RunningJob) bool {

func shouldBeReportedDone(job *job.RunningJob) bool {
for _, pod := range job.ActivePods {
if util.IsInTerminalState(pod) && !util.IsReportedDone(pod) {
if util.IsInTerminalState(pod) && util.HasCurrentStateBeenReported(pod) && !util.IsReportedDone(pod) {
return true
}
}
Expand Down

0 comments on commit 5dfb12a

Please sign in to comment.