MicroK8S OIDC Issue

[elielam]

Describe the bug

Microk8s OIDC not recognized.

Che version

7.47@latest

Steps to reproduce

Install Microk8s on debian 11.
Update kubeapi config for oidc.
Run chectl.

Expected behavior

Chectl must pass the oidc check.

Runtime

other (please specify in additional context)

Screenshots

No response

Installation method

chectl/latest

Environment

Linux

Eclipse Che Logs

2022-05-01T16:54:22.475Z Error: Command server:deploy failed. Error log: /home/agent/.cache/chectl/error.log. Eclipse Che logs: /tmp/chectl-logs/1651424061496.
2022-05-01T16:54:22.475Z     at newError (~/.local/share/chectl/client/7.47.0/lib/util.js:199:19)
2022-05-01T16:54:22.475Z     at Object.wrapCommandError (~/.local/share/chectl/client/7.47.0/lib/util.js:195:12)
2022-05-01T16:54:22.475Z     at Deploy.<anonymous> (~/.local/share/chectl/client/7.47.0/lib/commands/server/deploy.js:177:35)
2022-05-01T16:54:22.475Z     at Generator.throw (<anonymous>)
2022-05-01T16:54:22.475Z     at rejected (~/.local/share/chectl/client/7.47.0/node_modules/tslib/tslib.js:115:69)
2022-05-01T16:54:22.475Z Cause: Error: API server is not configured with OIDC Identity Provider, see details https://kubernetes.io/docs/reference/access-authn-authz/authentication/#configuring-the-api-server. To bypass OIDC Provider check, use '--skip-oidc-provider-check' flag
2022-05-01T16:54:22.475Z     at ~/.local/share/chectl/client/7.47.0/lib/commands/server/deploy.js:380:19
2022-05-01T16:54:22.475Z     at Generator.next (<anonymous>)
2022-05-01T16:54:22.475Z     at fulfilled (~/.local/share/chectl/client/7.47.0/node_modules/tslib/tslib.js:114:62)

Additional context

Hello,

I am currently having a problem installing eclipse che via chectl on microk8s.

The problem in question lies in the identification of the OIDC parameters of kubernetes.

I obviously (not without difficulty) took care to activate OIDC via keycloak by adding in /var/snap/microk8s/current/args/kube-apiserver the right arguments on my node and it works very well via kubectl.

However chectl fails to pass the verification step of my OIDC configuration.

I initially disabled the check via the --skip-oidc-provider-check argument but that only delayed the problem, the process finished on a timeout because the oauth-proxy container of the che-gateway pod crashed with the following message: "... oidc-issuer-url is required".

Looking closer in the code it appears that in the file we are looking in the kube-system namespace for a pod with the label component=kube-apiserver.

Here :

function ensureOIDCProviderInstalled(flags: any): Listr.ListrTask {
    ...
    const apiServerPods = await kube.getPodListByLabel('kube-system', 'component=kube-apiserver')

When I check in this namespace this is what I found :

The closest reference I find to what we are looking for is in the default namespace on the kubernetes service but i can't find to wich pod is it related.

Search pods with corresponding label :

Is my microk8s installation having a problem or am I missing something on the OIDC or chectl ?

Plateform : Linux servername 5.10.0-13-amd64 #1 SMP Debian 5.10.106-1 (2022-03-17) x86_64 GNU/Linux
Microk8s : Microk8s v1.21.11
Kubectl : v1.22.4
Kubernetes : v1.21.11-3+2bdf0a81ac1652

Thank you in advance (sorry for my bad english) !

[tolusha]

Hello @elielam

However chectl fails to pass the verification step of my OIDC configuration.

yes, pls use flag --skip-oidc-provider-check in this case. To be honest I have never tested it with microk8s. I would be nice if you help us to improve OIDC detection here.

the process finished on a timeout because the oauth-proxy container of the che-gateway pod crashed with the following message: "... oidc-issuer-url is required".

To make Eclipse Che working, you have to set the following fields in a CheCluster CR:

spec:
  auth:
    identityProviderURL: 
    oAuthClientName: 
    oAuthSecret: 

I have a look pls also:

• Devworkspace native auth on Kubernetes - Unauthorized! Configured service account doesn't have access #21049 (comment)
• Devworkspace native auth on Kubernetes - Unauthorized! Configured service account doesn't have access #21049 (comment)

/cc @sparkoo

[l0rd]

Added to the list of OIDC related issues

[che-bot]

Issues go stale after 180 days of inactivity. lifecycle/stale issues rot after an additional 7 days of inactivity and eventually close.

Mark the issue as fresh with /remove-lifecycle stale in a new comment.

If this issue is safe to close now please do so.

Moderators: Add lifecycle/frozen label to avoid stale mode.

[Mbd06b]

Hello @elielam

However chectl fails to pass the verification step of my OIDC configuration.

yes, pls use flag --skip-oidc-provider-check in this case. To be honest I have never tested it with microk8s. I would be nice if you help us to improve OIDC detection here.

the process finished on a timeout because the oauth-proxy container of the che-gateway pod crashed with the following message: "... oidc-issuer-url is required".

To make Eclipse Che working, you have to set the following fields in a CheCluster CR:

spec:
  auth:
    identityProviderURL: 
    oAuthClientName: 
    oAuthSecret: 

I have a look pls also:

• Devworkspace native auth on Kubernetes - Unauthorized! Configured service account doesn't have access #21049 (comment)
• Devworkspace native auth on Kubernetes - Unauthorized! Configured service account doesn't have access #21049 (comment)

/cc @sparkoo

For Microk8s
In addition to the keycloak values added to your checluster.yaml values spec
I needed to add the CHE_ODIC__USERNAME_CLAIM as follows ....

spec:
  components:
    cheServer:
      extraProperties:
         CHE_OIDC_USERNAME__CLAIM: email

Then I edited the API Server Configuration: Locate the configuration file for the Kubernetes API server (under /var/snap/microk8s/current/args/kube-apiserver for microk8s).

Add OIDC Options: Include the following lines (adjusting for your specific Keycloak configuration):

--oidc-issuer-url=https://<KEYCLOAK_DOMAIN>/auth/realms/<REALM_NAME>
--oidc-client-id=<CLIENT_ID>
--oidc-username-claim=email
--oidc-groups-claim=groups 

[Update] Just needed to add here, that if you are running a cluster with multiple nodes in microks you will need to apply the same --odic args to every node in your cluster, and manually restart each node for the oidc to apply correctly.

sudo microk8s stop && sudo microk8s start

[tolusha]

Thank you for your work.