[kube-prometheus-stack/grafana] It looks like variable expansion problems

[ryuseongryong]

Describe the bug a clear and concise description of what the bug is.

This is an issue I've found while doing a lot of repeat installs.

I'm using the authentication from grafana, which kube-prometheus-stack installs with, as the OIDC.
I'm not sure if it's a problem with grafana, which is used by recent versions of kube-prometheus-stack, or if it's a problem with kube-prometheus-stack.

The problem is with grafana's recognition of the client_secret in auth.generic_oauth in grafana.ini. I have confirmed that I am using grafana @ 7.0.* and it installs fine up to kube-prometheus-stack helm version 55.7.1.
However, from kube-prometheus-stack 55.8.0 onwards, we have confirmed that we are using grafana @ 7.1.*, and from this version onwards, the error occurs even though we are using variable expansion.

• client_secret: '$__file{/etc/secrets/auth_generic_oauth/client_secret}'(I checked that it is mounted in previous version)
• Error: INSTALLATION FAILED: execution error at (kube-prometheus-stack/charts/grafana/templates/deployment.yaml:36:28): Sensitive key 'auth.generic_oauth.client_secret' should not be defined explicitly in values. Use variable expansion instead.

What's your helm version?

version.BuildInfo{Version:"v3.14.0", GitCommit:"3fc9f4b2638e76f26739cd77c7017139be81d0ea", GitTreeState:"clean", GoVersion:"go1.21.6"}

What's your kubectl version?

Client Version: v1.29.0 Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3 Server Version: v1.29.0

Which chart?

prometheus-community/kube-prometheus-stack

What's the chart version?

after 55.8.0

What happened?

The problem is with grafana's recognition of the client_secret in auth.generic_oauth in grafana.ini. I have confirmed that I am using grafana @ 7.0.* and it installs fine up to kube-prometheus-stack helm version 55.7.1.
However, from kube-prometheus-stack 55.8.0 onwards, we have confirmed that we are using grafana @ 7.1.*, and from this version onwards, the error occurs even though we are using variable expansion.

• client_secret: '$__file{/etc/secrets/auth_generic_oauth/client_secret}'(I checked that it is mounted in previous version)
• Error: INSTALLATION FAILED: execution error at (kube-prometheus-stack/charts/grafana/templates/deployment.yaml:36:28): Sensitive key 'auth.generic_oauth.client_secret' should not be defined explicitly in values. Use variable expansion instead.

What you expected to happen?

installation with helm chart

How to reproduce it?

• client_secret: '$__file{/etc/secrets/auth_generic_oauth/client_secret}'(I checked that it is mounted in previous version)

Enter the changed values of values.yaml?

No response

Enter the command that you execute and failing/misfunctioning.

helm install --wait -f kube-prometheus-stack/values.yaml --version 55.11.0 kube-prometheus-stack prometheus-community/kube-prometheus-stack -n monitoring

Anything else we need to know?

kube-prometheus-stack 55.7.1 is OK(grafana 7.0.)
kube-prometheus-stack 55.8.0 is Error (grafana 7.1.)

[zeritti]

Please, see issue #2914 and PR #2867 (7.1) for reference in the Grafana chart repository.

[bdalpe]

PR open for fix: #2904

[bdalpe]

FYI -- the fix for the grafana chart is included in kube-prometheus-stack release 56.0.0.