[corechecks/containerlifecycle] Add Pod finished timestamp to containerlifecycle pod events

[sblumenthal]

NOTE: this PR is waiting on others to be merged in first, so some bits may change

What does this PR do?

This PR adds a pod exit timestamp to the PodEvents emitted by the container-lifecycle check, which is defined here

Motivation

Currently, we have the timestamp when a container terminates, but we do not have it on the pod side.

Additional Notes

This PR is waiting on the linked agent-payload PR to be merged, which itself is waiting on another agent-payload PR to be merged, which might affect the underlying data model used in this PR. As a result, some syntax might change between now and when the PR is marked as ready.

Possible Drawbacks / Trade-offs

We do not actually get the timestamp that a pod is killed by the kubelet, so to get around this I am using the timestamp of the first workloadmeta pull in which we did not see the pod anymore. The implementation currently sets all pods that were removed between intervals to have the same exited timestamp.

Describe how to test/QA your changes

Testing this is tricky, because (as far as I know) we do not have a way to inspect the payload generated by this check. I was able to validate this locally by building the agent with debug flags set, and then using delve to set breakpoints at the return statements of toPayloadModel and toEventModel in pkg/collector/corechecks/containerlifecycle/event.go. I was able to validate locally that the field is populated this way.

Reviewer's Checklist

• If known, an appropriate milestone has been selected; otherwise the Triage milestone is set.
• Use the major_change label if your change either has a major impact on the code base, is impacting multiple teams or is changing important well-established internals of the Agent. This label will be use during QA to make sure each team pay extra attention to the changed behavior. For any customer facing change use a releasenote.
• A release note has been added or the changelog/no-changelog label has been applied.
• Changed code has automated tests for its functionality.
• Adequate QA/testing plan information is provided if the qa/skip-qa label is not applied.
• At least one team/.. label has been applied, indicating the team(s) that should QA this change.
• If applicable, docs team has been notified or an issue has been opened on the documentation repo.
• If applicable, the need-change/operator and need-change/helm labels have been applied.
• If applicable, the k8s/<min-version> label, indicating the lowest Kubernetes version compatible with this feature.
• If applicable, the config template has been updated.

[maycmlee]

👍 for docs