Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kubeadm: Set the kubelet --resolv-conf flag conditionally on init #64665

Conversation

stealthybox
Copy link
Member

What this PR does / why we need it:
kubeadm init detects if systemd-resolved is running and configures the kubelet to use a working resolv.conf.
This patch also removes the warning message prompting manual user action for this configuration.

/area kubeadm
/area kubelet
/area dns
/kind bug
/priority important-soon

/sig cluster-lifecycle
/assign @timothysc

Which issue(s) this PR fixes
Fixes kubernetes/kubeadm#845

Special notes for your reviewer:
See the difference in KUBELET_KUBEADM_ARGS when running with this build and enabling the resolved daemon on Ubuntu 17.10:

root@vagrant:/vagrant/bin# bash << EOF
systemctl start systemd-resolved
./845_kubeadm init |& tail -n5
cat /var/lib/kubelet/kubeadm-flags.env

./845_kubeadm reset --force |& tail -n2
systemctl stop systemd-resolved
echo nameserver 8.8.8.8 > /etc/resolv.conf

./845_kubeadm init |& tail -n5
cat /var/lib/kubelet/kubeadm-flags.env

EOF
You can now join any number of machines by running the following on each node
as root:

  kubeadm join 10.0.2.15:6443 --token 77q84j.0342evur7rrfrwwx --discovery-token-ca-cert-hash sha256:190040f9c3adf8410bc6766dac79f8679870190564e15e8f8d1704fafa03f678

KUBELET_KUBEADM_ARGS=--cgroup-driver=cgroupfs --cni-bin-dir=/opt/cni/bin --cni-conf-dir=/etc/cni/net.d --network-plugin=cni --resolv-conf=/run/systemd/resolve/resolv.conf
I0603 06:49:16.592482   14106 reset.go:276] [reset] deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
I0603 06:49:16.592858   14106 reset.go:290] [reset] deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
You can now join any number of machines by running the following on each node
as root:

  kubeadm join 10.0.2.15:6443 --token 8mdart.gp67vq3nh9urq4z5 --discovery-token-ca-cert-hash sha256:da6b2e5841546eae134524b045e782f0dd91a6b53becc8d69c15d9eab9c88758

KUBELET_KUBEADM_ARGS=--cgroup-driver=cgroupfs --cni-bin-dir=/opt/cni/bin --cni-conf-dir=/etc/cni/net.d --network-plugin=cni

Release note:

`kubeadm init` detects if systemd-resolved is running and configures the kubelet to use a working resolv.conf.

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. area/kubeadm size/M Denotes a PR that changes 30-99 lines, ignoring generated files. area/kubelet area/dns kind/bug Categorizes issue or PR as related to a bug. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. labels Jun 3, 2018
@k8s-ci-robot k8s-ci-robot requested review from kad and timothysc June 3, 2018 07:08
@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jun 3, 2018
@dixudx
Copy link
Member

dixudx commented Jun 4, 2018

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 4, 2018
Copy link
Member

@luxas luxas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

Thanks @stealthybox 💯!

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dixudx, luxas, stealthybox

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 4, 2018
@luxas luxas added this to the v1.11 milestone Jun 4, 2018
@k8s-github-robot
Copy link

[MILESTONENOTIFIER] Milestone Pull Request: Up-to-date for process

@dixudx @stealthybox @timothysc

Pull Request Labels
  • sig/cluster-lifecycle: Pull Request will be escalated to these SIGs if needed.
  • priority/important-soon: Escalate to the pull request owners and SIG owner; move out of milestone after several unsuccessful escalation attempts.
  • kind/bug: Fixes a bug discovered during the current release.
Help

@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel comment for consistent failures.

Copy link
Member

@timothysc timothysc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I won't block it but perhaps we can do an audit of warnings and checks past freeze.

@@ -72,8 +73,12 @@ func buildKubeletArgMap(nodeRegOpts *kubeadmapi.NodeRegistrationOptions, registe
kubeletFlags["register-with-taints"] = strings.Join(taintStrs, ",")
}

if pids, _ := procfs.PidOf("systemd-resolved"); len(pids) > 0 {
// procfs.PidOf only returns an error if the regex is empty or doesn't compile, so we can ignore it
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO - we should still emit a message as a breadcrumb to the user in case anything happens.

@k8s-github-robot
Copy link

/test all [submit-queue is verifying that this PR is safe to merge]

@k8s-github-robot
Copy link

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here.

@k8s-github-robot k8s-github-robot merged commit 86ae84b into kubernetes:master Jun 4, 2018
@stealthybox stealthybox deleted the feature/kubeadm_845-systemd-resolved branch June 4, 2018 16:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/dns area/kubeadm area/kubelet cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Set the kubelet --resolv-conf flag conditionally at kubeadm init time
7 participants