Retry using a different pod when the node shutdown.

When the node shutdown then the retry pod will always be the same pod
because it can not recognise that the pod can not work anymore and
k8s can not delete the pod before the node recovers and it causes the retry is actually not to work.

I fix it by checking the DeletionTimestamp to know if the pod is actually
not work any more.

Fix #6558

Signed-off-by: yuzhipeng <zpyu@alauda.io>

pkg/pod/status.go

@@ -400,7 +400,8 @@ func DidTaskRunFail(pod *corev1.Pod) bool {
 			}
 		}
 	}
-	return false
+	// when the node shutdown the pod will be in Terminating and be deleted
+	return pod.DeletionTimestamp != nil
 }
 
 // IsPodArchived indicates if a pod is archived in the retriesStatus.