From a0d25ca5cd4166546388f8b544c519f191c807d0 Mon Sep 17 00:00:00 2001 From: Sergey Kanzhelev Date: Thu, 12 Sep 2024 23:51:22 +0000 Subject: [PATCH] clarifications for sidecar container feature --- .../sig-node/753-sidecar-containers/README.md | 105 +++--------------- keps/sig-node/753-sidecar-containers/kep.yaml | 5 +- 2 files changed, 18 insertions(+), 92 deletions(-) diff --git a/keps/sig-node/753-sidecar-containers/README.md b/keps/sig-node/753-sidecar-containers/README.md index 5b46edd0998..e1993199f7a 100644 --- a/keps/sig-node/753-sidecar-containers/README.md +++ b/keps/sig-node/753-sidecar-containers/README.md @@ -181,10 +181,10 @@ Items marked with (R) are required *prior to targeting to a milestone / release* - [X] (R) Design details are appropriately documented - [X] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors) - [X] e2e Tests for all Beta API Operations (endpoints) - - [ ] (R) Ensure GA e2e tests meet requirements for [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) - - [ ] (R) Minimum Two Week Window for GA e2e tests to prove flake free -- [ ] (R) Graduation criteria is in place - - [ ] (R) [all GA Endpoints](https://github.com/kubernetes/community/pull/1806) must be hit by [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) + - [X] (R) Ensure GA e2e tests meet requirements for [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) + - [X] (R) Minimum Two Week Window for GA e2e tests to prove flake free +- [X] (R) Graduation criteria is in place + - [X] (R) [all GA Endpoints](https://github.com/kubernetes/community/pull/1806) must be hit by [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) - [ ] (R) Production readiness review completed - [ ] (R) Production readiness review approved - [ ] "Implementation History" section is up-to-date for milestone @@ -401,8 +401,10 @@ startup will NOT be restarted and the whole Pod will fail. If Pod Once sidecar container is started (`postStart` completed and startup probe succeeded), this containers will be restarted even when the Pod `restartPolicy` -is `Never` or `OnFailure`. Furthermore, sidecar containers will be restarted -even during Pod termination. +is `Never` or `OnFailure`. + +Note, a separate KEP https://github.com/kubernetes/enhancements/issues/4438 will enable +sidecar containers to be restarted even during Pod termination. In order to minimize OOM kills of sidecar containers, the OOM adjustment for these containers will match or exceed the OOM score adjustment of regular @@ -1281,68 +1283,6 @@ to know in early stages of the KEP IMHO. ### Graduation Criteria - - #### Alpha - Feature implemented behind a feature flag @@ -1357,24 +1297,11 @@ in back-to-back releases. #### GA -- Allow to apply security policies on all containers in `initContainers` - collection. Example may be disabling `kubectl exec` on containers in - `initContainers` collection. +- All known issues are fixed +- Production use feedback addressed ### Upgrade / Downgrade Strategy - - #### Upgrade strategy Existing sidecars (implemented as regular containers) will still work as @@ -1612,9 +1539,10 @@ that might indicate a serious problem? - Labels:code, container_type (should be `init_container`) - Components exposing the metric: `kubelet-metrics` - Symptoms: high number of errors indicates that the kubelet is unable to start the sidecar containers -- [X] Events - - Event name: TBD - - Symptoms: high number of events indicates that the TGPS has been exceeded and sidecars have been terminated not gracefully +- [X] API objects + - Pods stuck in Pending state of Init container running. + - Type: API objects + - Symptoms: when the new field `restartPolicy:Always` was mistakenly stripped out by a webhook, Pod will get stuck. ###### Were upgrade and rollback tested? Was the upgrade->downgrade->upgrade path tested? @@ -1742,9 +1670,6 @@ Pick one more of these and delete the rest. - Type: Counter - Labels:code, container_type (should be `init_container`) - Components exposing the metric: `kubelet-metrics` -- [X] Events - - Event name: TBD - - should not appear, unless TGPS is exceeded and sidecars are terminated ###### Are there any missing metrics that would be useful to have to improve observability of this feature? @@ -1955,7 +1880,7 @@ Major milestones might include: - 2018-05-14: First proposal. - 2023-06-09: Target 1.28 for Alpha. - 2023-07-08: Alpha implementation merged. -- TODO: PRR completed and graduation to beta proposed. +- 1.29: feature is in Beta ## Drawbacks diff --git a/keps/sig-node/753-sidecar-containers/kep.yaml b/keps/sig-node/753-sidecar-containers/kep.yaml index 75c1e62f620..14be54d8ade 100644 --- a/keps/sig-node/753-sidecar-containers/kep.yaml +++ b/keps/sig-node/753-sidecar-containers/kep.yaml @@ -4,13 +4,14 @@ authors: - "@SergeyKanzhelev" - "@matthyx" - "@jpbetz" + - "@gjkim42" owning-sig: sig-node participating-sigs: - sig-scheduler - sig-apps status: implementable creation-date: 2018-05-14 -last-updated: 2024-01-25 +last-updated: 2024-09-12 reviewers: - "@mrunalp" # overall - "@ffromani" # resource management @@ -37,7 +38,7 @@ latest-milestone: "v1.29" milestone: alpha: "v1.28" beta: "v1.29" - stable: "v1.32" + stable: "v1.33" # The following PRR answers are required at alpha release # List the feature gate name and the components for which it must be enabled