Spec for workload cluster OIDC auth as the default auth method

[gusevda]

There are different aspects we need to touch to achieve workload cluster auth

Scope

We start with CAPA first, then look at other providers. The idea is that updating default apps will enable this.

Default Apps

• nginx ingress controller
• Dex
• athena
• cert-manager

We can add these to the default apps with sane defaults but we need to be mindful of existing apps installed
We could add a value to disable dex/oidc but we shouldn't really do it. Rather let customers migrate oidc setup first before updating to the new default apps version.
What about private clusters?
What about custom certs?

Dex operator

We need to solve the race condition issue with secret vs configmap connector configuration

Auth

We need to ensure that a clusterrolebinding is present for (giantswarm) admin group

Cluster App

We also have to think about the api server flags that need to be added, and restarting control plane nodes
We also have some customers that use oidc but do not use dex and we need to migrate them and make sure not to override existing settings

Login

We want to have a seamless login experience using kgs/opsctl

[anvddriesch]

Update:
Dex-operator manages configuration for WCs now, so let's flesh this one out.

[anvddriesch]

The spec points we came up with during refinement are all reflected as issues in the main epic here #2473
I will close this for now.