Add kubeRBACProxy property to support querying in cluster prometheus in openshift

[mittal-ishaan]

What does this PR change?

• Add kubeRBACProxy property to set KUBE_RBAC_PROXY_ENABLED as true in the env var of the cost-model.
• createPrometheusClusterRoleBinding property to create cluster role binding to grant required permissions to the serviceaccount to query prometheus with kube-rbac-proxy enabled.
• Add the role binding to grant permissions this external prometheus to scrape metrics from kubecost.
• Add a check that both kubeRBACProxy and BearerToken property can not be set together,

Does this PR rely on any other PRs?

opencost/opencost#2944

How does this PR impact users? (This is the kind of thing that goes in release notes!)

THis will enable openshift cluster users to configure installing kubecost while using their in-cluster prometheus.

Links to Issues or tickets this PR addresses or fixes

Closes #3690

What risks are associated with merging this PR? What is required to fully test this PR?

NA

How was this PR tested?

tested by installing kubecost using custom built cost-model image having these changes while disabling the bundled prometheus and using the in-cluster prometheus.

Have you made an update to documentation? If so, please provide the corresponding PR.

https://github.com/kubecost/docs/pull/1144/files

[jessegoodier]

This looks good to me. Let's merge when the opencost PR is merged.

[jessegoodier]

LGTM!

[thomasvn]

Is it safe to assume that the ServiceAccount we want to bind to will always be in openshift-monitoring?

[mittal-ishaan]

No, you are correct here. It is not safe to assume the namespace. I would have to correct this.

[thomasvn]

We should add these comments to the main values.yaml file as well!

[thomasvn]

If possible, I think we should also uncomment them and set default values when possible. Example:

  # Platforms is a higher-level abstraction for platform-specific values and settings.
  platforms:
    # Deploying to OpenShift (OCP) requires enabling this option.
    openshift:
      enabled: true  # Deploy Kubecost to OpenShift.
      createMonitoringClusterRoleBinding: false  # Create a Cluster Role Binding to allow using in-cluster prometheus or thanos.
      createMonitoringResourceReaderRoleBinding: false  # Create a Role and Role Binding to allow in-cluster prometheus or thanos to list and watch resources. This will be necessary if you are not using bundled prometheus and need to add scrape config for resources.
      monitoringServiceAccountName: prometheus-k8s  # Name of the service account to bind to the Resource Reader Role Binding.

[mittal-ishaan]

sure, will uncomment these. I added these specific to openshift and should make it more generalized. Should I move these to global instead? instead of having them under openshift?

[jessegoodier]

Good finds Thomas.

Ishaan- we merged because we have an urgent need with some customers and needed to test. Thank you for getting this done so quickly!

[thomasvn]

What ClusterRole is this referring to? I couldn't find any reference to this in the Helm chart. Is this an existing ClusterRole that's specific to OpenShift installs?

[mittal-ishaan]

yes its specific to openshift installs.
https://docs.redhat.com/en/documentation/openshift_container_platform/4.2/html/monitoring/cluster-monitoring#monitoring-accessing-prometheus-alerting-ui-grafana-using-the-web-console_accessing-prometheus

[thomasvn]

Awesome, thanks for clarifying! Lets leave a comment in the template?

[mittal-ishaan]

sure 👍

[mittal-ishaan]

Thank you Thomas for pointing these out.
I will create a PR to fix these today itself.