PodLifeTime: support CrashLoopBackOff container state

[a7i]

Use-case: We are trying to delete crashlooping pods after X seconds so they get re-created and hopefully fix themselves.

RemoveFailedPods plugin would not work because a Pod in CrashLoopBackOff has a pod phase of Running

  containerStatuses:
  - containerID: containerd://274b89ca9b2d7a815cf6550453af29663c114c51917358bb02330613d39fbb73
    lastState:
      terminated:
        containerID: containerd://274b89ca9b2d7a815cf6550453af29663c114c51917358bb02330613d39fbb73
        exitCode: 0
        finishedAt: "2023-06-06T22:09:03Z"
        reason: Completed
        startedAt: "2023-06-06T22:09:03Z"
    name: descheduler-podlifetime
    ready: false
    restartCount: 3
    started: false
    state:
      waiting:
        message: back-off 40s restarting failed container=rdescheduler-podlifetime pod=descheduler-podlifetime
        reason: CrashLoopBackOff
  phase: Running

[a7i]

/cc @knelasevero @damemi @ingvagabund

[damemi]

/approve

I think there is potential for some confusion between RemoveFailedPods and this though. I agree that it is technically correct to put it here because a pod in CrashLoopBackoff is technically running. But users may not see it that way.

Maybe add a note in the RemoveFailedPods section saying something like "Because pods in CrashLoopBackoff are technically Running, they can be evicted using PodLifetime"

Also as a follow up it might be useful to add a restartThreshold parameter for this specifically to evict pods that have crashlooped X times instead of crashlooping for X seconds

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: damemi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [damemi]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ingvagabund]

/lgtm