Proposal: kubelet authentication/authorization #32518

liggitt · 2016-09-12T20:19:05Z

Proposal for kubernetes/enhancements#89

This change is

liggitt · 2016-09-12T20:29:55Z

cc @kubernetes/sig-auth @kubernetes/sig-node @roberthbailey @mikedanese @cjcullen @erictune @kayrus @deads2k

luxas · 2016-09-12T20:31:45Z

cc @kubernetes/sig-cluster-lifecycle

timothysc · 2016-09-12T20:38:09Z

docs/proposals/kubelet-auth.md

+
+This proposal assumes the existence of:
+
+* a functioning API server


Isn't this a chicken-egg supposition. /cc @mikedanese

Nope, you don't need to call the kubelet API to bootstrap an apiserver with a kubelet

k8s-bot · 2016-09-12T20:57:55Z

GCE e2e build/test passed for commit cf6efb2aabdae41e6816eed84299003a4892c5b7.

philips · 2016-09-12T22:42:58Z

This is something we want to get fixed up in v1.5. cc @ericchiang @aaronlevy @yifan-gu @euank @pbx0

smarterclayton · 2016-09-13T04:19:57Z

docs/proposals/kubelet-auth.md

+
+### Anonymous
+
+Add a new `--anonymous-auth=[true|false]` option to the kubelet.


I know this is in use in various places, is this the first time this is part of Kube?

part of #32386 for the apiserver

smarterclayton · 2016-09-13T04:21:28Z

Looks fairly similar to what works already in OpenShift - the other flags look consistent with our current uses in apiserver. Bootstrapping may in the future want to play in this mode, although I prefer bootstrapping to be a pre-stage to full kubelet startup and the kubelet to be required to transition from bootstrap into one of these modes explicitly.

Automatic merge from submit-queue Allow webhook authenticator to use TokenReviewsInterface Refactors the authentication webhook to be able to be fed a kubeconfig file or a TokenReviewsInterface Fixes a bug with webhooks not retrying on 429/500 errors Related to #32518 (comment)

erictune · 2016-09-26T17:32:29Z

@alex-mohr @roberthbailey @cjcullen
I think you are interested in kubelet API security.

jbeda · 2016-09-26T17:41:39Z

docs/proposals/kubelet-auth.md

+The master API server can already be started with `--kubelet-client-certificate` and
+`--kubelet-client-key` options in order to make authenticated requests to the kubelet.
+
+### Bearer token


Is a bearer token method really necessary? Are there many scenarios for users accessing the Kubelet API or can we lock this down even further?

in openshift, we've been able to eliminate the unsecured kubelet port, and have heapster gather stats and metrics data using a service account token against the secured kubelet port

Yeah, I'm for this particular part of the proposal staying in. Being able to centrally administer what has access to these endpoints is great. And the tooling around issuance and revocation of service accounts is much more built out than our PKI and CRL infrastructure.

smarterclayton · 2016-09-27T20:25:43Z

The overall proposal accurately represents most of the security discussions we had when locking the Kubelet down in OpenShift and seems broadly in line with our existing apiserver flags and direction. This proposal LGTM - can folks on this proposal who have concerns still do a pass and raise those soon so we can move forward?

ericchiang · 2016-10-04T17:11:11Z

docs/proposals/kubelet-auth.md

+
+## API Changes
+
+None


Possible new resources for the SubjectAccessReview?

no API changes are needed to support the SubjectAccessReviews the kubelet would submit (it would express the checks in verb and group/version/resource/subresource/name terms)

ericchiang · 2016-10-04T17:14:03Z

@smarterclayton I'm +1 on this.

yifan-gu · 2016-10-04T18:38:41Z

LGTM

smarterclayton · 2016-10-04T19:44:55Z

@erictune any last comments?

erictune · 2016-10-05T00:21:45Z

I'd really like one of @cjcullen @alex-mohr @roberthbailey to take a look at this because this relies on SubjectAccessReview and TokenReviews, which we don't expose on GKE.

cjcullen · 2016-10-05T18:35:49Z

GKE does not enable the SubjectAccessReview and TokenReview APIs because we don't really want to support those as generic Kubernetes APIs. I'm not completely opposed to providing this as an option for kubelet auth, but it feels fishy to me.

We don't want to expose the TokenReview endpoint on GKE, because it could possibly allow 3rd parties to inspect https://www.googleapis.com/auth/cloud-platform OAuth2 tokens.

I don't think overloading SubjectAccessReview for kubelet authz is a great idea. We would be crossing the namespaces of kubelet resources and kubernetes resources. SubjectAccessReview is currently pretty purpose-built for the user-facing Kubernetes API. Do we see the kubelet API becoming more of a user-facing thing? If not, SubjectAccessReview feels like overkill.

I think the x509 option provides the cleanest way forward for GKE. As a first step, we could mint a cert for kube-apiserver and a cert for heapster with a separate CA than the one we use for the apiserver's certs, and then AllowAll inside the kubelet. This would add another place to worry about cert rotation, and would make a compromised heapster a potential avenue to long-lived access to any pod. To mitigate that a bit, we could make authz give read-write to kube-apiserver and read-only to heapster.

LGTM to the proposal (to unblock any work that is waiting). I don't see the webhook options as viable paths forward for GKE (without rethinking other things), but I think the other options here will get us going the right direction. Thanks @liggitt for putting this together.

smarterclayton · 2016-10-05T20:34:24Z

I'll say our view is somewhat biased because the vast majority of deployed
clusters we've seen lack anything like central google oauth, and if they
did have one, they'd want Kube to manage the cluster infra auth (i.e. they
want the rules applying to kubelet access to be centrally managed on the
cluster). if you don't have a worldwide auth domain, you want your Kube
cluster to be internally consistent and not have separate authorization
paths.

The kubelet API today is user facing in two directions - administrative
level access (logs and metrics) and to heapster. Heapster having
administrative access to nodes is concerning, so at least some role
granularity is required. I suspect other system agents in the future will
require access (monitoring like Kube-snap).

I don't think it's unreasonable that the authorizer on the kubelet have
multiple paths (which include cert auth and SAR) and different deployments
get to make that choice, just like the masters.

On Oct 4, 2016, at 8:22 PM, Eric Tune notifications@github.com wrote:

I'd really like one of @cjcullen https://github.com/cjcullen @alex-mohr
https://github.com/alex-mohr @roberthbailey
https://github.com/roberthbailey to take a look at this because this
relies on SubjectAccessReview and TokenReviews, which we don't expose on
GKE.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#32518 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABG_p4fnkdelS-4XG3XkK4RnNHHmuIQlks5qwu2mgaJpZM4J6_J8
.

smarterclayton · 2016-10-07T21:54:28Z

If no other comments, LGTM. Thanks

jbeda

Small tweak. Hopefully not disruptive.

jbeda · 2016-10-07T22:20:07Z

docs/proposals/kubelet-auth.md

+
+### x509 client certificate
+
+Add a new `--client-ca-file=[file]` option to the kubelet.


Quick comment -- any place where we have a CA certificate we should support a bundle or a set of certificates. Otherwise it is almost impossible to do key rotation.

We should also make sure that we plan to have this stuff driven by component configs.

Agree. File could be a bundle, just like the API server. Also, we'd hook up the CRL bits here as soon as we have them worked out for the apiserver authenticator.

k8s-github-robot · 2016-10-07T22:22:11Z

Automatic merge from submit-queue

Automatic merge from submit-queue Allow webhook authorizer to use SubjectAccessReviewInterface Refactors the authorization webhook to be able to be fed a kubeconfig file or a SubjectAccessReviewsInterface Added tests to exercise retry logic, and ensure the correct serialized version is sent to the remote webhook (I also made sure the new tests passed on the current webhook impl in master) c.f. #32547 c.f. #32518

Automatic merge from submit-queue Proposal: kubelet authentication/authorization Proposal for kubernetes/enhancements#89

Automatic merge from submit-queue Allow webhook authorizer to use SubjectAccessReviewInterface Refactors the authorization webhook to be able to be fed a kubeconfig file or a SubjectAccessReviewsInterface Added tests to exercise retry logic, and ensure the correct serialized version is sent to the remote webhook (I also made sure the new tests passed on the current webhook impl in master) c.f. kubernetes/kubernetes#32547 c.f. kubernetes/kubernetes#32518

googlebot added the cla: yes label Sep 12, 2016

liggitt mentioned this pull request Sep 12, 2016

Authenticated/Authorized access to kubelet API kubernetes/enhancements#89

Closed

22 tasks

k8s-github-robot assigned smarterclayton Sep 12, 2016

k8s-github-robot added kind/design Categorizes issue or PR as related to design. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. release-note-label-needed labels Sep 12, 2016

timothysc reviewed Sep 12, 2016
View reviewed changes

aaronlevy mentioned this pull request Sep 12, 2016

Kubelet TLS Bootstrap kubernetes/enhancements#43

Closed

liggitt mentioned this pull request Sep 13, 2016

Allow webhook authenticator to use TokenReviewsInterface #32547

Merged

smarterclayton reviewed Sep 13, 2016
View reviewed changes

liggitt added this to the v1.5 milestone Sep 21, 2016

kubelet authentication/authorization proposal

b626a82

liggitt added release-note-none Denotes a PR that doesn't merit a release note. and removed release-note-label-needed labels Sep 22, 2016

liggitt force-pushed the kubelet-auth-proposal branch from cf6efb2 to b626a82 Compare September 22, 2016 19:06

jbeda reviewed Sep 26, 2016

View reviewed changes

ericchiang reviewed Oct 4, 2016

View reviewed changes

This was referenced Oct 5, 2016

Allow webhook authorizer to use SubjectAccessReviewInterface #34071

Merged

Secure master -> node communication #11816

Closed

smarterclayton added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 7, 2016

jbeda reviewed Oct 7, 2016

View reviewed changes

k8s-github-robot merged commit f93e01d into kubernetes:master Oct 7, 2016

liggitt deleted the kubelet-auth-proposal branch October 8, 2016 01:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Proposal: kubelet authentication/authorization #32518

Proposal: kubelet authentication/authorization #32518

liggitt commented Sep 12, 2016 •

edited

Loading

liggitt commented Sep 12, 2016

luxas commented Sep 12, 2016

timothysc Sep 12, 2016

liggitt Sep 12, 2016 •

edited

Loading

k8s-bot commented Sep 12, 2016

philips commented Sep 12, 2016

smarterclayton Sep 13, 2016

liggitt Sep 13, 2016

smarterclayton commented Sep 13, 2016

erictune commented Sep 26, 2016

jbeda Sep 26, 2016

liggitt Sep 26, 2016

ericchiang Oct 4, 2016

smarterclayton commented Sep 27, 2016

ericchiang Oct 4, 2016

liggitt Oct 4, 2016

ericchiang commented Oct 4, 2016

yifan-gu commented Oct 4, 2016

smarterclayton commented Oct 4, 2016

erictune commented Oct 5, 2016

cjcullen commented Oct 5, 2016

smarterclayton commented Oct 5, 2016

smarterclayton commented Oct 7, 2016

jbeda left a comment

jbeda Oct 7, 2016

liggitt Oct 7, 2016

k8s-github-robot commented Oct 7, 2016


		This proposal assumes the existence of:

		* a functioning API server


		### Anonymous

		Add a new `--anonymous-auth=[true\|false]` option to the kubelet.


		### x509 client certificate

		Add a new `--client-ca-file=[file]` option to the kubelet.

Proposal: kubelet authentication/authorization #32518

Proposal: kubelet authentication/authorization #32518

Conversation

liggitt commented Sep 12, 2016 • edited Loading

liggitt commented Sep 12, 2016

luxas commented Sep 12, 2016

Choose a reason for hiding this comment

liggitt Sep 12, 2016 • edited Loading

Choose a reason for hiding this comment

k8s-bot commented Sep 12, 2016

philips commented Sep 12, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smarterclayton commented Sep 13, 2016

erictune commented Sep 26, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smarterclayton commented Sep 27, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ericchiang commented Oct 4, 2016

yifan-gu commented Oct 4, 2016

smarterclayton commented Oct 4, 2016

erictune commented Oct 5, 2016

cjcullen commented Oct 5, 2016

smarterclayton commented Oct 5, 2016

smarterclayton commented Oct 7, 2016

jbeda left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

k8s-github-robot commented Oct 7, 2016

liggitt commented Sep 12, 2016 •

edited

Loading

liggitt Sep 12, 2016 •

edited

Loading