From ee449f3435251495cb5fbe5e9edc5100912dfd3d Mon Sep 17 00:00:00 2001 From: "Lubomir I. Ivanov" Date: Mon, 16 May 2022 19:36:17 +0300 Subject: [PATCH] sig-cl/kubeadm/1739: update KEP for support of patching kubelet config - Generalize the KEP to not only target static pods for the control plane - Add "kubeletconfiguration" target --- .../1739-customization-with-patches/README.md | 71 +++++++++++++------ 1 file changed, 48 insertions(+), 23 deletions(-) diff --git a/keps/sig-cluster-lifecycle/kubeadm/1739-customization-with-patches/README.md b/keps/sig-cluster-lifecycle/kubeadm/1739-customization-with-patches/README.md index 42b24bb4184c..44df8c6522e4 100644 --- a/keps/sig-cluster-lifecycle/kubeadm/1739-customization-with-patches/README.md +++ b/keps/sig-cluster-lifecycle/kubeadm/1739-customization-with-patches/README.md @@ -168,11 +168,12 @@ A good summary is probably at least a paragraph in length. This KEP proposes a replacement for the feature introduced by the kubeadm KEP "20190722-Advanced-configurations-with-kubeadm-(Kustomize)". -It has the same scope and goals, allowing the users to amend manifests -generated by kubeadm using patches, but not using Kustomize as the backend -for applying the patches. +It has similar scope and goals, allowing the users to amend manifests +and configuration files generated by kubeadm using patches, but not using +Kustomize as the backend for applying the patches. -The existing Kustomize implementation in kubeadm is in Alpha state. +The Kustomize implementation in kubeadm was in an Alpha state and was +removed. ## Motivation @@ -198,10 +199,10 @@ know that this has succeeded? --> - Implement a solution for applying patches over kubeadm generated manifests -during the common kubeadm commands "init", "join" and "upgrade" and their phases. +and configuration during the common kubeadm commands "init", "join" and "upgrade" +and their phases. - Deprecate and remove the existing Kustomize solution after a grace period of at least one release. -- To initially allow patching of static Pod manifests only. ### Non-Goals @@ -214,8 +215,6 @@ and make progress. sane security values. - Allow patching of core addon configuration generated by kubeadm - e.g. CoreDNS, kube-proxy (until further notice). -- Allowing patches to be added using the kubeadm ComponentConfig (until -further notice). - Replace the kubeadm ComponentConfig. The sane defaults would still be generated by kubeadm. @@ -265,6 +264,11 @@ liveness probes for edge clusters. As a cluster administrator, I want to upgrade my cluster preserving all the patches that were applied during "init"/"join". +#### Story 4 + +As a cluster administrator, I want to be able to apply instance specific +configuration to kubelets on different nodes. + ### Notes/Constraints/Caveats (optional) +This section covers the `corev1.Pod` as a example target for version skew. + Once introduced, the feature will support patching at minimum the `corev1.Pod` object. If at some point the Kubernetes core API graduates to `v2`, kubeadm would have to support patching both `corev2` and `corev1` for at least one release @@ -584,8 +607,7 @@ This is due to the fact kubeadm supports deploying Kubernetes `v1.YY` and `v1.YY And if `corev2` is added in `v1.YY`, kubeadm must be able to handle `corev1` that is supported in `v1.YY-1`. -If the patching mechanic one day supports patching different API types, -it must also apply similar skew strategy to those types. +A similar skew strategy must be applied for other supported patch target API types. ## Production Readiness Review Questionnaire @@ -624,7 +646,7 @@ _This section must be completed when targeting alpha to a release._ - [x] Other - Describe the mechanism: When a new kubeadm control-plane Node joins the cluster it can optionally - apply custom patches to the static Pods that kubeadm configures. + apply custom patches to kubeadm generated configuration. - Will enabling / disabling the feature require downtime of the control plane? No, unless the user created patches result in bad configuration and @@ -646,7 +668,7 @@ _This section must be completed when targeting alpha to a release._ feature, can it break the existing applications?). Yes. Once the patches are applied for a `kubeadm init|join|upgrade` command, the user can decide to roll-back the changes which will result in a restart -of the control-plane component static Pods that were patches on this Node. +of component that was customized by the patches. * **What happens if we reenable the feature if it was previously rolled back?** This is supported by invoking kubeadm phases. The patches will re-apply @@ -669,7 +691,7 @@ _This section must be completed when targeting beta graduation to a release._ Try to be as paranoid as possible - e.g. what if some components will restart in the middle of rollout? The feature can result in malfunctioning control-plane components, due to the -fact that it targets patching of static Pod manifests with user values. +fact that it targets patching of configuration with user values. The user is responsible for configuring the control-plane correctly. * **What specific metrics should inform a rollback?** @@ -744,7 +766,7 @@ _This section must be completed when targeting beta graduation to a release._ - Usage description: - Impact of its outage on the feature: - Impact of its degraded performance or high error rates on the feature: -No, this feature customizes static Pod manifests for the control-plane. +No, this feature customizes on top of kubeadm generated configuration. ### Scalability @@ -793,8 +815,8 @@ No. operations covered by [existing SLIs/SLOs][]?** Think about adding additional work or introducing new steps in between (e.g. need to do X to start a container), etc. Please describe the details. -No, unless the user patches the control-plane static Pods with undesired -configuration. +No, unless the user patches kubeadm generated configuration with undesired +settings. * **Will enabling / using this feature result in non-negligible increase of resource usage (CPU, RAM, disk, IO, ...) in any components?** @@ -803,8 +825,8 @@ configuration. volume), significant amount of data send and/or received over network, etc. This through this both in small and large cases, again with respect to the [supported limits][]. -No, unless the user patches the control-plane static Pods with undesired -configuration. +No, unless the user patches kubeadm generated configuration with undesired +settings. ### Troubleshooting @@ -859,6 +881,9 @@ Major milestones might include - 2020-05-14: addressed feedback, filled PRR questionnaire, KEP marked as implementable - 2021-09-09: marked the feature as graduated to Beta. This was done as part of the kubeadm v1beta3 API work. The actions under "Alpha -> Beta" are completed. +- 2022-05-16: adapted the KEP to be more generic, allowing support for patching +any kubeadm generated configuration in the future and not only control plane component +manifests. Added enumeration of supported patch targets. ## Drawbacks