Move resolve.conf setting to Kubelet config file

[NeilW]

BUG REPORT

Versions

kubeadm version (use kubeadm version):

# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.3", GitCommit:"2e7996e3e2712684bc73f0dec0200d64eec7fe40", GitTreeState:"clean", BuildDate:"2020-05-20T12:49:29Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

Environment:

• Kubernetes version (use kubectl version):

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.3", GitCommit:"2e7996e3e2712684bc73f0dec0200d64eec7fe40", GitTreeState:"clean", BuildDate:"2020-05-20T12:52:00Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.3", GitCommit:"2e7996e3e2712684bc73f0dec0200d64eec7fe40", GitTreeState:"clean", BuildDate:"2020-05-20T12:43:34Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

• Cloud provider or hardware configuration:
  Brightbox
• OS (e.g. from /etc/os-release)::
  Ubuntu 20.04
• Kernel (e.g. uname -a):
  inux srv-c3x7t 5.4.0-33-generic discuss the meaning of kubeadm init --api-advertise-addresses #37-Ubuntu SMP Thu May 21 12:53:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
• Others:

What happened?

kubelet is logging a deprecation error on both the control plane and the worker nodes

Flag --resolv-conf has been deprecated, This parameter should be set via the config file

What you expected to happen?

The "resolvConf" entry in the KubeletConfiguration should be set to the correct value and the flag removed from /var/lib/kubelet/kubeadm-flags.env

How to reproduce it (as minimally and precisely as possible)?

Install k8s master node using kubeadm, add in a worker using kubeadm join.

Anything else we need to know?

Upgraded kubelet files from 1.16.X have a resolvConf: /etc/resolv.conf entry in the config file, which is currently overridden by the command line flag. Upgrades may be doing things differently from installs.

[neolit123]

hello, this seems already resolved by:
kubernetes/kubernetes#90394
?

xref #2111

/kind feature
/priority awaiting-more-evidence

[neolit123]

^ also note for 1.18 we have "action required" for users to manually remove the flag from existing files.

[NeilW]

Doesn't appear to be. This is the result of a fresh 1.18 install.

[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet

ubuntu@srv-9nzt6:~$ sudo cat /var/lib/kubelet/kubeadm-flags.env
KUBELET_KUBEADM_ARGS="--cloud-provider=external --container-runtime=remote --container-runtime-endpoint=/run/containerd/containerd.sock --resolv-conf=/run/systemd/resolve/resolv.conf"
ubuntu@srv-9nzt6:~$ sudo cat /var/lib/kubelet/config.yaml
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
  anonymous:
    enabled: false
  webhook:
    cacheTTL: 0s
    enabled: true
  x509:
    clientCAFile: /etc/kubernetes/pki/ca.crt
authorization:
  mode: Webhook
  webhook:
    cacheAuthorizedTTL: 0s
    cacheUnauthorizedTTL: 0s
cgroupDriver: systemd
clusterDNS:
- 172.30.0.10
clusterDomain: cluster.local
cpuManagerReconcilePeriod: 0s
evictionPressureTransitionPeriod: 0s
fileCheckFrequency: 0s
healthzBindAddress: 127.0.0.1
healthzPort: 10248
httpCheckFrequency: 0s
imageMinimumGCAge: 0s
kind: KubeletConfiguration
nodeStatusReportFrequency: 0s
nodeStatusUpdateFrequency: 0s
rotateCertificates: true
runtimeRequestTimeout: 0s
staticPodPath: /etc/kubernetes/manifests
streamingConnectionIdleTimeout: 0s
syncFrequency: 0s
volumeStatsAggPeriod: 0s

[neolit123]

@NeilW the fix will be in 1.19 and we cannot backport to 1.18.

[NeilW]

ok