✨ Add protection to metrics endpoint using authn/authz via controller-runtime feature #4003

camilamacedo86 · 2024-07-04T22:44:43Z

This PR re-introduces authentication and authorization protection for the metrics endpoint using WithAuthenticationAndAuthorization provided by controller-runtime.

We are re-introducing this protection, which is similar to what was previously done via kube-rbac-proxy, whose usage was discontinued in the project.

PS.:: Address item 3 mapped in : #3871

Author: Joe Lanford joe.lanford@gmail.com
Co-Author: Camila Macedo

…untime feature This PR re-introduce authn/authz protection for the endpoint but without use the kube-rbac-proxy project and image. Signed-off-by: Joe Lanford <joe.lanford@gmail.com>

camilamacedo86 · 2024-07-04T23:00:19Z

/skip APIDiff

/override APIDiff

k8s-ci-robot · 2024-07-04T23:00:21Z

@camilamacedo86: /override requires failed status contexts, check run or a prowjob name to operate on.
The following unknown contexts/checkruns were given:

APIDiff

Only the following failed contexts/checkruns were expected:

EasyCLA
netlify/kubebuilder/deploy-preview
pull-kubebuilder-e2e-k8s-1-28-7
pull-kubebuilder-e2e-k8s-1-29-2
pull-kubebuilder-e2e-k8s-1-30-0
pull-kubebuilder-test
tide

If you are trying to override a checkrun that has a space in it, you must put a double quote on the context.

In response to this:

/skip APIDiff

/override APIDiff

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Kavinjsir

lgtm 👍🏼

Kavinjsir · 2024-07-05T03:21:41Z

pkg/plugins/golang/v4/scaffolds/internal/templates/main.go

+			// These configurations ensure that only authorized users and service accounts
+			// can access the metrics endpoint. The RBAC are configured in 'config/rbac/kustomization.yaml'. More info:
+			// https://pkg.go.dev/sigs.k8s.io/controller-runtime@{{ .ControllerRuntimeVersion }}/pkg/metrics/filters#WithAuthenticationAndAuthorization
+			FilterProvider: filters.WithAuthenticationAndAuthorization,


Nits:
Similar to the concern in the previous PR, would it be necessary/good to remind user of assigning the cluster role to the clients with the permission to scrape the metrics?
See the ref:

To scrape metrics e.g. via Prometheus the client needs a ClusterRole with the following rule: * nonResourceURLs: "/metrics", verbs: get

I think that the problem is that depends on how the Prometheus is configured.
We might could say something like Prometheus requires access to the metrics endpoint.
But we are either linking the metrics doc so users can check that, there we are clarifying it right.

Anyway, lets to do one thing:
Since it is a nit, lets move forward with as it is, however, if you see any good way to improve the comments/docs etc, please feel free to push a PR with your idea and we can shape things around.

varshaprasad96

I did a quick check by checking out the branch and running the example locally. It does work from my end.

Thank you for the contribution!

/lgtm
/approve

k8s-ci-robot · 2024-07-05T07:40:43Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: camilamacedo86, Kavinjsir, varshaprasad96

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [Kavinjsir,camilamacedo86,varshaprasad96]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

sbueringer · 2024-07-15T10:48:34Z

docs/book/src/cronjob-tutorial/testdata/project/cmd/main.go

+			// These configurations ensure that only authorized users and service accounts
+			// can access the metrics endpoint. The RBAC are configured in 'config/rbac/kustomization.yaml'. More info:
+			// https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.18.4/pkg/metrics/filters#WithAuthenticationAndAuthorization
+			FilterProvider: filters.WithAuthenticationAndAuthorization,


I think this should only be set if secureMetrics is true.

By always enabling this folks have to always provide a ServiceAccount token. But these tokens should never be send via HTTP

we can address this one good catcher.

Images provided under gcr.io/kubebuilder/ will be unavailable from March 18, 2025. Projects initialized with Kubebuilder versions v3.14 or lower utilize gcr.io/kubebuilder/kube-rbac-proxy to protect the metrics endpoint. Following the work in kubernetes-sigs/kubebuilder#4003, this commit removes the kube-rbac-proxy container and let the main container of the controller expose the metrics via HTTPS and by using the WithAuthenticatoinAndAuthorization filter. This also includes a minor fix in BuildService escaped during the resolution of some conflicts during a rebase. Related to kubernetes-sigs/kubebuilder#3871

…low-up to PR kubernetes-sigs#4243) This commit is a follow-up to PR kubernetes-sigs#4243, which introduced support for using cert-manager certificates for securing the metrics endpoint and ServiceMonitor. Related to kubernetes-sigs#3871 and kubernetes-sigs#4003 Key enhancements: - Added support for configuring certificate integration via a Kustomize patch. - Introduced configurable flags for greater flexibility in customization. - Use Certwatcher to allow certificate rotation This configuration provides an option for users to be production-ready

…low-up to PR kubernetes-sigs#4243) feat/fix: enhance cert-manager integration for metrics endpoints This commit is a follow-up to PR kubernetes-sigs#4243, which introduced support for using cert-manager certificates for securing the metrics endpoint and ServiceMonitor. Related to kubernetes-sigs#3871 and kubernetes-sigs#4003 Key enhancements: - Added support for configuring certificate integration via a Kustomize patch. - Introduced configurable flags for greater flexibility in customization. - Use Certwatcher to allow certificate rotation This configuration provides an option for users to be production-ready. These improvements enhance usability and adaptability while maintaining compatibility with the initial implementation. As the feature has not yet been released, this update ensures a polished and user-friendly integration for upcoming releases.

…low-up to PR kubernetes-sigs#4243) This commit is a follow-up to PR kubernetes-sigs#4243, which introduced support for using cert-manager certificates for securing the metrics endpoint and ServiceMonitor. Related to kubernetes-sigs#3871 and kubernetes-sigs#4003 Key enhancements: - Added support for configuring certificate integration via a Kustomize patch. - Introduced configurable flags for greater flexibility in customization. - Use Certwatcher to allow certificate rotation These improvements enhance usability and adaptability while maintaining compatibility with the initial implementation. As the feature has not yet been released, this update ensures a polished and user-friendly integration for upcoming releases.

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jul 4, 2024

k8s-ci-robot requested review from rashmigottipati and varshaprasad96 July 4, 2024 22:44

k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jul 4, 2024

camilamacedo86 mentioned this pull request Jul 4, 2024

✨ Re-introduce authn/authz for metrics endpoint #3968

Closed

camilamacedo86 requested a review from joelanford July 4, 2024 22:48

Add protection to metrics endpoint using authn/authz via controller-r…

3f67dcb

…untime feature This PR re-introduce authn/authz protection for the endpoint but without use the kube-rbac-proxy project and image. Signed-off-by: Joe Lanford <joe.lanford@gmail.com>

camilamacedo86 force-pushed the pr-3968 branch from 170c73a to 3f67dcb Compare July 4, 2024 22:50

camilamacedo86 requested review from erikgb and Kavinjsir July 4, 2024 22:56

Kavinjsir approved these changes Jul 5, 2024

View reviewed changes

varshaprasad96 approved these changes Jul 5, 2024

View reviewed changes

k8s-ci-robot assigned varshaprasad96 Jul 5, 2024

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 5, 2024

camilamacedo86 merged commit de1cc60 into kubernetes-sigs:master Jul 5, 2024
19 of 21 checks passed

camilamacedo86 mentioned this pull request Jul 5, 2024

[META - Phases]: Discontinue Kube RBAC Proxy in Default Kubebuilder Scaffolding #3871

Closed

5 tasks

pmalek mentioned this pull request Jul 5, 2024

Removed support for kube-rbac-proxy in Kubebuilder v4 Kong/gateway-operator#399

Open

2 tasks

sbueringer reviewed Jul 15, 2024

View reviewed changes

camilamacedo86 mentioned this pull request Jul 17, 2024

Conditionally Enable filters.WithAuthenticationAndAuthorization Only When secureMetrics is True #4020

Closed

camilamacedo86 mentioned this pull request Dec 1, 2024

✨ (go/v4): feat/fix: enhance cert-manager integration for metrics endpoints (follow-up to PR #4243) #4400

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

✨ Add protection to metrics endpoint using authn/authz via controller-runtime feature #4003

✨ Add protection to metrics endpoint using authn/authz via controller-runtime feature #4003

camilamacedo86 commented Jul 4, 2024 •

edited

Loading

camilamacedo86 commented Jul 4, 2024

k8s-ci-robot commented Jul 4, 2024

Kavinjsir left a comment

Kavinjsir Jul 5, 2024

camilamacedo86 Jul 5, 2024

varshaprasad96 left a comment

k8s-ci-robot commented Jul 5, 2024

sbueringer Jul 15, 2024

camilamacedo86 Jul 15, 2024

✨ Add protection to metrics endpoint using authn/authz via controller-runtime feature #4003

✨ Add protection to metrics endpoint using authn/authz via controller-runtime feature #4003

Conversation

camilamacedo86 commented Jul 4, 2024 • edited Loading

camilamacedo86 commented Jul 4, 2024

k8s-ci-robot commented Jul 4, 2024

Kavinjsir left a comment

Choose a reason for hiding this comment

Kavinjsir Jul 5, 2024

Choose a reason for hiding this comment

camilamacedo86 Jul 5, 2024

Choose a reason for hiding this comment

varshaprasad96 left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Jul 5, 2024

sbueringer Jul 15, 2024

Choose a reason for hiding this comment

camilamacedo86 Jul 15, 2024

Choose a reason for hiding this comment

camilamacedo86 commented Jul 4, 2024 •

edited

Loading