Add support for configmap of headers for auth-url per ingress #4550

actgardner · 2019-09-10T14:28:37Z

What this PR does / why we need it:
Adds a new annotation nginx.ingress.kubernetes.io/auth-proxy-set-headers which specifies a ConfigMap of header: value which are passed to an external authentication service when nginx.ingress.kubernetes.io/auth-url is set.

This is valuable to reduce the use of snippets to set headers for each ingress. The ingress validates that the keys and values will produce a valid nginx configuration.

k8s-ci-robot · 2019-09-10T14:28:38Z

Welcome @actgardner!

It looks like this is your first PR to kubernetes/ingress-nginx 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/ingress-nginx has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

k8s-ci-robot · 2019-09-10T14:28:44Z

Hi @actgardner. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

aledbf · 2019-09-10T14:39:30Z

/ok-to-test

codecov-io · 2019-09-10T14:52:12Z

Codecov Report

❗ No coverage uploaded for pull request base (master@cb2889b). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master    #4550   +/-   ##
=========================================
  Coverage          ?   58.56%           
=========================================
  Files             ?       88           
  Lines             ?     6736           
  Branches          ?        0           
=========================================
  Hits              ?     3945           
  Misses            ?     2356           
  Partials          ?      435

Impacted Files	Coverage Δ
internal/ingress/controller/template/template.go	`79.15% <100%> (ø)`
internal/ingress/annotations/authreq/main.go	`75.38% <100%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cb2889b...786a3b6. Read the comment docs.

rootfs/etc/nginx/template/nginx.tmpl

ElvinEfendi · 2019-09-10T15:16:23Z

Please also add e2e test in https://github.com/kubernetes/ingress-nginx/blob/master/test/e2e/annotations/auth.go to assert the behaviour.

test/e2e-image/overlay/kustomization.yaml

aledbf · 2019-09-10T18:44:43Z

@actgardner two things, first, squash the commits and you can test this locally running FOCUS="Annotations.*Auth" make kind-e2e-test to avoid triggering multiple times the e2e tests in Prow

actgardner · 2019-09-13T14:13:34Z

test/e2e/annotations/auth.go

+
+			annotations := map[string]string{
+				"nginx.ingress.kubernetes.io/auth-url":               "http://foo.bar/basic-auth/user/password",
+				"nginx.ingress.kubernetes.io/auth-proxy-set-headers": "auth-headers",


@ElvinEfendi @aledbf This e2e test isn't working - it doesn't seem to be adding the external auth section to the nginx config at all. I've poked around the e2e framework and I'm not sure why this is failing but the snippet tests above succeed? Is there something obvious I'm missing?

Are you sure auth-headers configmap gets created? Can you assert the existence of it here? Also why do you not create it programmatically like we do in other e2e tests. When you create it dynamically you can also assert behaviour when auth-proxy-set-headers is configured but configmap does not exist.

actgardner · 2019-09-17T19:41:49Z

internal/ingress/controller/store/store.go

@@ -529,44 +529,39 @@ func New(
 		AddFunc: func(obj interface{}) {
 			cm := obj.(*corev1.ConfigMap)
 			key := k8s.MetaNamespaceKey(cm)
-			// updates to configuration configmaps can trigger an update
-			if key == configmap || key == tcp || key == udp {


@ElvinEfendi These were the issue - the Store only notified the ingress of updates to a small set of ConfigMaps that are specified using flags when the controller is started (the global, tcp and udp ConfigMaps).

This fixes my issue (the end-to-end tests pass now), but I'm worried about the impact of this change on a noisy cluster (one where there's a lot of ConfigMap changes)? It would definitely have some performance impact. It seems like the alternative would be to rearchitect how the Store works so it supports a whitelist that can be updated on the fly, which is a big and fairly complex, risky change.

@actgardner yes, we restrict when to update to avoid too many invocations to Sync(), which could not trigger any change in nginx but increase the CPU and memory utilization

I have a POC to only listen to one object at the time https://gist.github.com/aledbf/030f90fd8fe087048782fe8d0cfd9ce2 but I never finished the integration. With this approach, we can just watch those configmaps we really use and not everything

aledbf · 2019-09-17T22:41:22Z

/retest

internal/ingress/controller/store/store.go

aledbf · 2019-09-22T21:18:23Z

@actgardner any chance you can address the comments before next Friday?
We plan to release 0.26, and I want to include this change in the release.

actgardner · 2019-09-23T13:31:03Z

@aledbf I can try to address the feedback before Friday, it's going to conflict with this PR though? #4586

What's the actual desired end-state here?

yes, we restrict when to update to avoid too many invocations to Sync(), which could not trigger any change in nginx but increase the CPU and memory utilization

Should I rearchitect this to maintain a whitelist of all the headers configmaps? The change in #4586 also seems like it's going to call Sync on every configmap update?

aledbf · 2019-09-23T14:14:12Z

Should I rearchitect this to maintain a whitelist of all the headers configmaps? The change in #4586 also seems like it's going to call Sync on every configmap update?

No, we will call the sync only if the configmap is referenced by an annotation

aledbf · 2019-09-23T14:14:51Z

You should just rebase the PR, removing the changes in store.go

aledbf · 2019-09-24T14:44:33Z

@actgardner You can rebase now. Also, please squash the commits

ElvinEfendi · 2019-09-24T15:38:14Z

/approve

aledbf · 2019-09-24T16:11:01Z

/lgtm

k8s-ci-robot · 2019-09-24T16:11:19Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: actgardner, aledbf, ElvinEfendi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [ElvinEfendi,aledbf]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

aledbf · 2019-09-24T16:12:15Z

@actgardner thank you and apologies for the delay

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Sep 10, 2019

k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Sep 10, 2019

k8s-ci-robot requested review from aledbf and ElvinEfendi September 10, 2019 14:29

actgardner force-pushed the upstream-auth-proxy-set-headers branch from 0d74783 to bd8d4f5 Compare September 10, 2019 14:32

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 10, 2019

actgardner force-pushed the upstream-auth-proxy-set-headers branch from 8630bde to 5833c5d Compare September 10, 2019 14:59

ElvinEfendi reviewed Sep 10, 2019

View reviewed changes

rootfs/etc/nginx/template/nginx.tmpl Outdated Show resolved Hide resolved

actgardner force-pushed the upstream-auth-proxy-set-headers branch 2 times, most recently from a71b1ff to 66a2bda Compare September 10, 2019 17:24

aledbf reviewed Sep 10, 2019

View reviewed changes

test/e2e-image/overlay/kustomization.yaml Outdated Show resolved Hide resolved

actgardner force-pushed the upstream-auth-proxy-set-headers branch from 2963175 to 3a3eac3 Compare September 10, 2019 18:41

actgardner force-pushed the upstream-auth-proxy-set-headers branch 2 times, most recently from 97e51e6 to dd4db29 Compare September 13, 2019 12:56

actgardner commented Sep 13, 2019

View reviewed changes

actgardner force-pushed the upstream-auth-proxy-set-headers branch from dd4db29 to 7e1574c Compare September 17, 2019 19:38

actgardner commented Sep 17, 2019

View reviewed changes

aledbf reviewed Sep 17, 2019

View reviewed changes

internal/ingress/controller/store/store.go Outdated Show resolved Hide resolved

aledbf reviewed Sep 17, 2019

View reviewed changes

internal/ingress/controller/store/store.go Outdated Show resolved Hide resolved

aledbf mentioned this pull request Sep 22, 2019

Nginx fails to listen to tcp and udp services when no ingress is defined #4585

Closed

actgardner force-pushed the upstream-auth-proxy-set-headers branch from 7e1574c to 89d59e4 Compare September 24, 2019 13:46

Add support for configmap of headers to be sent to external auth service

786a3b6

actgardner force-pushed the upstream-auth-proxy-set-headers branch from 6f02a17 to 786a3b6 Compare September 24, 2019 14:54

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 24, 2019

k8s-ci-robot assigned aledbf Sep 24, 2019

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 24, 2019

k8s-ci-robot merged commit 1dc4d18 into kubernetes:master Sep 24, 2019

ElvinEfendi mentioned this pull request Sep 24, 2019

sort auth proxy headers from configmap #4596

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for configmap of headers for auth-url per ingress #4550

Add support for configmap of headers for auth-url per ingress #4550

actgardner commented Sep 10, 2019

k8s-ci-robot commented Sep 10, 2019

k8s-ci-robot commented Sep 10, 2019

aledbf commented Sep 10, 2019

codecov-io commented Sep 10, 2019 •

edited

Loading

ElvinEfendi commented Sep 10, 2019

aledbf commented Sep 10, 2019

actgardner Sep 13, 2019

ElvinEfendi Sep 16, 2019 •

edited

Loading

actgardner Sep 17, 2019

aledbf Sep 17, 2019 •

edited

Loading

aledbf commented Sep 17, 2019

aledbf commented Sep 22, 2019

actgardner commented Sep 23, 2019

aledbf commented Sep 23, 2019

aledbf commented Sep 23, 2019

aledbf commented Sep 24, 2019

ElvinEfendi commented Sep 24, 2019

aledbf commented Sep 24, 2019

k8s-ci-robot commented Sep 24, 2019

aledbf commented Sep 24, 2019

Add support for configmap of headers for auth-url per ingress #4550

Add support for configmap of headers for auth-url per ingress #4550

Conversation

actgardner commented Sep 10, 2019

k8s-ci-robot commented Sep 10, 2019

k8s-ci-robot commented Sep 10, 2019

aledbf commented Sep 10, 2019

codecov-io commented Sep 10, 2019 • edited Loading

Codecov Report

ElvinEfendi commented Sep 10, 2019

aledbf commented Sep 10, 2019

actgardner Sep 13, 2019

Choose a reason for hiding this comment

ElvinEfendi Sep 16, 2019 • edited Loading

Choose a reason for hiding this comment

actgardner Sep 17, 2019

Choose a reason for hiding this comment

aledbf Sep 17, 2019 • edited Loading

Choose a reason for hiding this comment

aledbf commented Sep 17, 2019

aledbf commented Sep 22, 2019

actgardner commented Sep 23, 2019

aledbf commented Sep 23, 2019

aledbf commented Sep 23, 2019

aledbf commented Sep 24, 2019

ElvinEfendi commented Sep 24, 2019

aledbf commented Sep 24, 2019

k8s-ci-robot commented Sep 24, 2019

aledbf commented Sep 24, 2019

codecov-io commented Sep 10, 2019 •

edited

Loading

ElvinEfendi Sep 16, 2019 •

edited

Loading

aledbf Sep 17, 2019 •

edited

Loading