Implement support for ManagedCertificate CRD #508

krzykwas · 2018-10-02T16:06:09Z

This change adds to Ingress support for ManagedCertificate CRD to integrate with https://github.com/GoogleCloudPlatform/gke-managed-certs

k8s-ci-robot · 2018-10-02T16:06:12Z

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.

If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.
If you have done the above and are still having issues with the CLA being reported as unsigned, please email the CNCF helpdesk: helpdesk@rt.linuxfoundation.org

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

krzykwas · 2018-10-02T16:23:56Z

I signed it

krzykwas · 2018-10-02T16:34:41Z

I signed it

krzykwas · 2018-10-02T16:36:54Z

/assign @bowei

rramkumar1 · 2018-10-02T16:53:58Z

/ok-to-test

krzykwas · 2018-10-10T16:42:19Z

/assign @rramkumar1

cmd/glbc/main.go

rramkumar1 · 2018-10-10T17:26:38Z

pkg/controller/controller_test.go

@@ -43,6 +44,13 @@ var (
 	clusterUID      = "aaaaa"
 )

+type recorderProducerMock struct {
+}
+


Can you move this mock to pkg/events.

In the same file as the interface definition is fine.

rramkumar1 · 2018-10-10T17:31:00Z

pkg/loadbalancers/utils.go

+	separator = ","
+)
+
+// splitAnnotation splits annotation by separator and trims whitespace


Can you put this in pkg/utils instead?

Done. I also added a unit test. There was one special case I didn't expect: if annotation was empty, splitAnnotation produced []string{""} instead of nil or []string{}, I fixed it.

pkg/loadbalancers/certificates.go

bowei · 2018-10-15T05:28:59Z

can you rebase and squash the commits, then split into:

vendor/ only changes
files changed for vendoring
implementation
unit tests

krzykwas · 2018-10-15T13:00:46Z

I rebased, squashed and split the commits into vendor/+Gopkg.lock changes, implementation and unit tests. I didn't get what files changed for vendoring means.

rramkumar1 · 2018-10-18T16:39:45Z

/cc @prameshj

k8s-ci-robot · 2018-10-18T16:39:46Z

@rramkumar1: GitHub didn't allow me to request PR reviews from the following users: prameshj.

Note that only kubernetes members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @prameshj

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

pkg/utils/annotation.go

rramkumar1 · 2018-10-19T23:11:51Z

pkg/loadbalancers/certificates.go

 func (l *L7) checkSSLCert() error {
-	// Handle Pre-Shared cert and early return if used
-	if used, err := l.usePreSharedCert(); used {
+	// Handle annotation managed-certificates


So the previous behavior was that if the user specified both pre-shared certs and secrets, we would only take the pre-shared certs and ignore the secrets. Correct me if I am wrong, but here it seems like you accept all three (managed, pre-shared and secret).

I would prefer to stick with the existing semantics. Specifically, Managed certs should take precedence over everything else.

I think that the current code either uses managed certificates and pre-shared-cert or managed certificates and k8s secrets. I think it does not support all 3 types at once. This is because of return at line 51.:

if used { l.sslCerts = append(managedSslCerts, preSharedSslCerts...) return err }

Also I added a test case for that to loadbalancer_test.go (one of last two test cases).

Any reason we want to mix the modes? If we support both managed certs and preshared together, user might be confused why preshared and secrets don't work together or why all 3 are not picked up. Supporting just one mode with a priority in case more than one is specified might be clearer?

I guess one reason to combine preshared and managed certs is because neither of them is ingress created and hence do not need to be garbage-collected. I am curious to see if you had any other reasons, and also motivation for supporting managed certs and secrets .

Only with modes combined it is possible to support no-downtime migration scenarios. GCLB has its own logic for selecting a certificate and Ingress really should just pass all the certificates down to GCLB. But pre-shared-cert and secrets are already implemented differently and I just wanted to keep this behavior unchanged.

As discussed offline, I think it would make sense to keep consistency with the existing behavior. Namely, only take one form of cert. In this case, the precedence would be managed cert > pre-shared > secret.

prameshj · 2018-10-23T06:16:35Z

pkg/events/events.go

@@ -0,0 +1,32 @@
+/*
+Copyright 2015 The Kubernetes Authors.


nit : Change to 2018?

ACK - please leave it unresolved and once we agree what changes are necessary I'll take a look to fix it.

prameshj · 2018-10-23T06:20:35Z

pkg/loadbalancers/certificates.go

 func (l *L7) checkSSLCert() error {
-	// Handle Pre-Shared cert and early return if used
-	if used, err := l.usePreSharedCert(); used {
+	// Handle annotation managed-certificates


Any reason we want to mix the modes? If we support both managed certs and preshared together, user might be confused why preshared and secrets don't work together or why all 3 are not picked up. Supporting just one mode with a priority in case more than one is specified might be clearer?

prameshj · 2018-10-23T06:23:25Z

pkg/loadbalancers/certificates.go

 func (l *L7) checkSSLCert() error {
-	// Handle Pre-Shared cert and early return if used
-	if used, err := l.usePreSharedCert(); used {
+	// Handle annotation managed-certificates


I guess one reason to combine preshared and managed certs is because neither of them is ingress created and hence do not need to be garbage-collected. I am curious to see if you had any other reasons, and also motivation for supporting managed certs and secrets .

prameshj · 2018-10-23T06:35:08Z

pkg/loadbalancers/certificates.go

+// getExistingSecretsSslCerts fetches SslCertificate resources created and managed by this load balancer
+// instance. These SslCertificate resources were created based on kubernetes secrets in Ingress
+// configuration.
+func (l *L7) getExistingSecretsSslCerts() ([]*compute.SslCertificate, error) {


Maybe call this "getIngressManagedSslCerts()" ?

ACK, I can rename later when introducing all required changes.

prameshj · 2018-10-23T06:44:53Z

pkg/utils/annotation.go

+	var result []string
+	for _, token := range strings.Split(annotation, separator) {
+		if token != "" {
+			result = append(result, strings.TrimSpace(token))


Is it possible for token to be " ", in which case we will still append an empty string to result since trimspace will make it "" ?
Maybe just check the result of TrimSpace for empty string?

Good catch, thanks. I'll also fix it once we agree what changes are necessary.

pkg/utils/annotation_test.go

pkg/loadbalancers/l7.go

…e-managed-certs

rramkumar1 · 2018-10-29T17:41:15Z

/lgtm
/hold

@prameshj if this looks good to you, we can merge.

prameshj

Some small nits, looks good to me otherwise.
Thanks for making the changes.

pkg/loadbalancers/certificates.go

prameshj · 2018-10-29T17:59:40Z

pkg/loadbalancers/certificates.go

+	}
+
+	sel := labels.NewSelector()
+	sel.Add(*req)


Can req be nil if there were no errors in line 156?

I think it can't based on the implementation of apimachinery labels, and also based on the Go convention for constructors I observed: err != nil or err == nil and an object is instantiated.

ok, thanks for verifying.

prameshj · 2018-10-29T18:03:18Z

pkg/loadbalancers/certificates.go

+	for _, mcrt := range mcrts {
+		if mcrt.Status.CertificateName != "" {
+			names = append(names, mcrt.Status.CertificateName)
+		}


Is it valid to have managed cert status with empty Certificate name? Should we log an error in this case?

When a user creates a ManagedCertificate, he wouldn't usually fill in the status field, so the CertificateName field can be empty and it is not an error.

…SslCertificate resources

rramkumar1 · 2018-10-30T15:37:43Z

/lgtm
/hold cancel

k8s-ci-robot · 2018-10-30T15:37:58Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: krzykwas, rramkumar1

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [rramkumar1]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

fproulx-dfuse · 2018-11-19T14:57:24Z

@krzykwas when can we expect this to be available in prod ?
Also, how and when are those releases made available to Kubernetes Engine, do I need to wait for an update of my Master for it to be available ?

k8s-ci-robot added the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label Oct 2, 2018

k8s-ci-robot requested review from freehan and thockin October 2, 2018 16:06

k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Oct 2, 2018

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Oct 2, 2018

k8s-ci-robot assigned bowei Oct 2, 2018

k8s-ci-robot removed the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Oct 2, 2018

This was referenced Oct 10, 2018

No frontend configured on load balancer GoogleCloudPlatform/gke-managed-certs#2

Closed

Ingress annotation fails GoogleCloudPlatform/gke-managed-certs#3

Closed

k8s-ci-robot assigned rramkumar1 Oct 10, 2018

rramkumar1 reviewed Oct 10, 2018

View reviewed changes

krzykwas force-pushed the mcrt branch 2 times, most recently from 00571a9 to 934d510 Compare October 15, 2018 12:53

rramkumar1 reviewed Oct 19, 2018

View reviewed changes

prameshj reviewed Oct 23, 2018

View reviewed changes

krzykwas added 3 commits October 26, 2018 18:53

Implement support for ManagedCertificate CRD: vendor dependency to gk…

2bea160

…e-managed-certs

Implement support for ManagedCertificate CRD: implementation

b8eae9f

Implement support for ManagedCertificate CRD: unit tests

35be9b2

krzykwas force-pushed the mcrt branch from 934d510 to 35be9b2 Compare October 26, 2018 21:19

k8s-ci-robot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lgtm "Looks good to me", indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Oct 29, 2018

prameshj reviewed Oct 29, 2018

View reviewed changes

Implement support for ManagedCertificate CRD: rename method creating …

aa9d103

…SslCertificate resources

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 30, 2018

k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. and removed do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. labels Oct 30, 2018

k8s-ci-robot merged commit 5a27ec5 into kubernetes:master Oct 30, 2018

Implement support for ManagedCertificate CRD #508

Implement support for ManagedCertificate CRD #508

Conversation

krzykwas commented Oct 2, 2018

k8s-ci-robot commented Oct 2, 2018

krzykwas commented Oct 2, 2018

krzykwas commented Oct 2, 2018

krzykwas commented Oct 2, 2018

rramkumar1 commented Oct 2, 2018

krzykwas commented Oct 10, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

krzykwas Oct 11, 2018 • edited Loading

Choose a reason for hiding this comment

bowei commented Oct 15, 2018

krzykwas commented Oct 15, 2018

rramkumar1 commented Oct 18, 2018

k8s-ci-robot commented Oct 18, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rramkumar1 commented Oct 29, 2018

prameshj left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rramkumar1 commented Oct 30, 2018

k8s-ci-robot commented Oct 30, 2018

fproulx-dfuse commented Nov 19, 2018

krzykwas Oct 11, 2018 •

edited

Loading