Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RBAC enabled but not restricting user #8310

Open
3 tasks done
LoricAndre opened this issue Apr 4, 2022 · 12 comments
Open
3 tasks done

RBAC enabled but not restricting user #8310

LoricAndre opened this issue Apr 4, 2022 · 12 comments
Labels
area/sso-rbac type/bug type/security Security related

Comments

@LoricAndre
Copy link
Contributor

LoricAndre commented Apr 4, 2022 •
edited by agilgur5
Loading

Checklist

  • Double-checked my configuration.
  • Tested using the latest version.
  • Used the Emissary executor.

Summary

What happened/what you expected to happen?
After setting up SSO and RBAC, SSO is working and assigning me the right ServiceAccount, but the rights I have on the server are more open than the associated role gives.
This is my RBAC SA/Role/RoleBinding:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: argo-workflow-default-user-login
  namespace: {{ .Release.Namespace }}
  annotations:
    workflows.argoproj.io/rbac-rule: "'my_group' in groups"
    workflows.argoproj.io/rbac-rule-precedence: "0"  # if the user is not in my_group, he should not and cannot access the server
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: argo-workflow-reader
  namespace: {{ .Release.Namespace }}
rules:
  - apiGroups:
      - argoproj.io
    resources:
      - workflows
      - workfloweventbindings
      - workflowtemplates
      - cronworkflows
      - cronworkflows/finalizers
    verbs:
      - get
      - list
      - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: argo-workflow-reader
  namespace: {{ .Release.Namespace }}
subjects:
- kind: ServiceAccount
  name: argo-workflow-default-user-login
roleRef:
  kind: Role
  name: argo-workflow-reader
  apiGroup: rbac.authorization.k8s.io

What version are you running?
v3.3.1

Diagnostics

Paste the smallest workflow that reproduces the bug. We must be able to run the workflow.

Any workflow
# Logs for the workflow controller:

time="2022-04-04T12:05:38.101Z" level=info msg="Update leases 200"
time="2022-04-04T12:05:41.374Z" level=info msg="Watch workflowtemplates 200"
time="2022-04-04T12:05:43.117Z" level=info msg="Get leases 200"
time="2022-04-04T12:05:43.127Z" level=info msg="Update leases 200"
time="2022-04-04T12:05:48.138Z" level=info msg="Get leases 200"
time="2022-04-04T12:05:48.147Z" level=info msg="Update leases 200"
time="2022-04-04T12:05:52.939Z" level=info msg="List workflows 200"
time="2022-04-04T12:05:52.939Z" level=info msg=healthz age=5m0s err="<nil>" instanceID= labelSelector="!workflows.argoproj.io/phase,!workflows.argoproj.io/controller-instanceid" managedNamespace=
time="2022-04-04T12:05:53.160Z" level=info msg="Get leases 200"
time="2022-04-04T12:05:53.169Z" level=info msg="Update leases 200"
time="2022-04-04T12:05:55.580Z" level=info msg="List workflowtasksets 404"
E0404 12:05:55.580889       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.21.5/tools/cache/reflector.go:167: Failed to watch *v1alpha1.WorkflowTaskSet: failed to list *v1alpha1.WorkflowTaskSet: the server could not find the requested resource (get workflowtasksets.argoproj.io)
time="2022-04-04T12:05:57.967Z" level=info msg="Watch configmaps 200"
time="2022-04-04T12:05:58.179Z" level=info msg="Get leases 200"
time="2022-04-04T12:05:58.189Z" level=info msg="Update leases 200"
time="2022-04-04T12:06:03.205Z" level=info msg="Get leases 200"
time="2022-04-04T12:06:03.216Z" level=info msg="Update leases 200"
time="2022-04-04T12:06:08.225Z" level=info msg="Get leases 200"
time="2022-04-04T12:06:08.237Z" level=info msg="Update leases 200"
time="2022-04-04T12:06:13.249Z" level=info msg="Get leases 200"
time="2022-04-04T12:06:13.257Z" level=info msg="Update leases 200"
time="2022-04-04T12:06:18.272Z" level=info msg="Get leases 200"
time="2022-04-04T12:06:18.284Z" level=info msg="Update leases 200"
time="2022-04-04T12:06:23.294Z" level=info msg="Get leases 200"
time="2022-04-04T12:06:23.304Z" level=info msg="Update leases 200"
time="2022-04-04T12:06:28.316Z" level=info msg="Get leases 200"
time="2022-04-04T12:06:28.328Z" level=info msg="Update leases 200"
time="2022-04-04T12:06:32.497Z" level=info msg="Watch clusterworkflowtemplates 200"
time="2022-04-04T12:06:33.341Z" level=info msg="Get leases 200"
time="2022-04-04T12:06:33.355Z" level=info msg="Update leases 200"
time="2022-04-04T12:06:33.671Z" level=info msg="Watch workflows 200"
time="2022-04-04T12:06:38.367Z" level=info msg="Get leases 200"
time="2022-04-04T12:06:38.378Z" level=info msg="Update leases 200"
time="2022-04-04T12:06:43.391Z" level=info msg="Get leases 200"
time="2022-04-04T12:06:43.405Z" level=info msg="Update leases 200"
time="2022-04-04T12:06:47.182Z" level=info msg="List workflowtasksets 404"
E0404 12:06:47.182314       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.21.5/tools/cache/reflector.go:167: Failed to watch *v1alpha1.WorkflowTaskSet: failed to list *v1alpha1.WorkflowTaskSet: the server could not find the requested resource (get workflowtasksets.argoproj.io)
time="2022-04-04T12:06:48.415Z" level=info msg="Get leases 200"
time="2022-04-04T12:06:48.427Z" level=info msg="Update leases 200"
time="2022-04-04T12:06:52.939Z" level=info msg="List workflows 200"
time="2022-04-04T12:06:52.939Z" level=info msg=healthz age=5m0s err="<nil>" instanceID= labelSelector="!workflows.argoproj.io/phase,!workflows.argoproj.io/controller-instanceid" managedNamespace=
time="2022-04-04T12:06:53.440Z" level=info msg="Get leases 200"
time="2022-04-04T12:06:53.451Z" level=info msg="Update leases 200"
time="2022-04-04T12:06:58.404Z" level=info msg="Watch configmaps 200"
time="2022-04-04T12:06:58.463Z" level=info msg="Get leases 200"
time="2022-04-04T12:06:58.473Z" level=info msg="Update leases 200"
time="2022-04-04T12:07:03.485Z" level=info msg="Get leases 200"
time="2022-04-04T12:07:03.497Z" level=info msg="Update leases 200"
time="2022-04-04T12:07:08.512Z" level=info msg="Get leases 200"
time="2022-04-04T12:07:08.522Z" level=info msg="Update leases 200"
time="2022-04-04T12:07:13.535Z" level=info msg="Get leases 200"
time="2022-04-04T12:07:13.558Z" level=info msg="Update leases 200"
time="2022-04-04T12:07:18.568Z" level=info msg="Get leases 200"
time="2022-04-04T12:07:18.579Z" level=info msg="Update leases 200"
time="2022-04-04T12:07:19.450Z" level=info msg="Watch pods 200"
time="2022-04-04T12:07:19.584Z" level=info msg="List workflowtasksets 404"
E0404 12:07:19.584489       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.21.5/tools/cache/reflector.go:167: Failed to watch *v1alpha1.WorkflowTaskSet: failed to list *v1alpha1.WorkflowTaskSet: the server could not find the requested resource (get workflowtasksets.argoproj.io)
time="2022-04-04T12:07:23.592Z" level=info msg="Get leases 200"
time="2022-04-04T12:07:23.605Z" level=info msg="Update leases 200"
time="2022-04-04T12:07:28.613Z" level=info msg="Get leases 200"
time="2022-04-04T12:07:28.623Z" level=info msg="Update leases 200"
time="2022-04-04T12:07:33.635Z" level=info msg="Get leases 200"
time="2022-04-04T12:07:33.646Z" level=info msg="Update leases 200"
time="2022-04-04T12:07:38.657Z" level=info msg="Get leases 200"
time="2022-04-04T12:07:38.668Z" level=info msg="Update leases 200"
time="2022-04-04T12:07:43.680Z" level=info msg="Get leases 200"
time="2022-04-04T12:07:43.690Z" level=info msg="Update leases 200"
time="2022-04-04T12:07:48.702Z" level=info msg="Get leases 200"
time="2022-04-04T12:07:48.725Z" level=info msg="Update leases 200"
time="2022-04-04T12:07:52.935Z" level=info msg="List workflows 200"
time="2022-04-04T12:07:52.935Z" level=info msg=healthz age=5m0s err="<nil>" instanceID= labelSelector="!workflows.argoproj.io/phase,!workflows.argoproj.io/controller-instanceid" managedNamespace=
time="2022-04-04T12:07:53.735Z" level=info msg="Get leases 200"
time="2022-04-04T12:07:53.758Z" level=info msg="Update leases 200"
time="2022-04-04T12:07:57.285Z" level=info msg="Alloc=6497 TotalAlloc=7311249 Sys=74065 NumGC=3513 Goroutines=202"
time="2022-04-04T12:07:58.772Z" level=info msg="Get leases 200"
time="2022-04-04T12:07:58.782Z" level=info msg="Update leases 200"
time="2022-04-04T12:08:03.342Z" level=info msg="List workflowtasksets 404"
E0404 12:08:03.342876       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.21.5/tools/cache/reflector.go:167: Failed to watch *v1alpha1.WorkflowTaskSet: failed to list *v1alpha1.WorkflowTaskSet: the server could not find the requested resource (get workflowtasksets.argoproj.io)
time="2022-04-04T12:08:03.794Z" level=info msg="Get leases 200"
time="2022-04-04T12:08:03.813Z" level=info msg="Update leases 200"
time="2022-04-04T12:08:08.827Z" level=info msg="Get leases 200"
time="2022-04-04T12:08:08.837Z" level=info msg="Update leases 200"
time="2022-04-04T12:08:13.848Z" level=info msg="Get leases 200"
time="2022-04-04T12:08:13.857Z" level=info msg="Update leases 200"
time="2022-04-04T12:08:14.561Z" level=info msg="Queueing Succeeded workflow argo-workflow/lovely-python-wjg8h for delete in 1m38s"
time="2022-04-04T12:10:45.411Z" level=info msg="List workflowtasksets 404"
E0404 12:10:45.411898       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.21.5/tools/cache/reflector.go:167: Failed to watch *v1alpha1.WorkflowTaskSet: failed to list *v1alpha1.WorkflowTaskSet: the server could not find the requested resource (get workflowtasksets.argoproj.io)
time="2022-04-04T12:10:49.588Z" level=info msg="Get leases 200"
time="2022-04-04T12:10:49.601Z" level=info msg="Update leases 200"
time="2022-04-04T12:10:52.935Z" level=info msg="List workflows 200"
time="2022-04-04T12:10:52.935Z" level=info msg=healthz age=5m0s err="<nil>" instanceID= labelSelector="!workflows.argoproj.io/phase,!workflows.argoproj.io/controller-instanceid" managedNamespace=
time="2022-04-04T12:10:54.612Z" level=info msg="Get leases 200"
time="2022-04-04T12:10:54.624Z" level=info msg="Update leases 200"
time="2022-04-04T12:10:59.645Z" level=info msg="Get leases 200"
time="2022-04-04T12:10:59.657Z" level=info msg="Update leases 200"
time="2022-04-04T12:11:04.671Z" level=info msg="Get leases 200"
time="2022-04-04T12:11:04.682Z" level=info msg="Update leases 200"
time="2022-04-04T12:11:09.694Z" level=info msg="Get leases 200"
time="2022-04-04T12:11:09.722Z" level=info msg="Update leases 200"
time="2022-04-04T12:11:14.741Z" level=info msg="Get leases 200"
time="2022-04-04T12:11:14.751Z" level=info msg="Update leases 200"
time="2022-04-04T12:11:19.764Z" level=info msg="Get leases 200"
time="2022-04-04T12:11:19.781Z" level=info msg="Update leases 200"
time="2022-04-04T12:11:24.794Z" level=info msg="Get leases 200"
time="2022-04-04T12:11:24.804Z" level=info msg="Update leases 200"
time="2022-04-04T12:11:29.818Z" level=info msg="Get leases 200"
time="2022-04-04T12:11:29.831Z" level=info msg="Update leases 200"
time="2022-04-04T12:11:34.839Z" level=info msg="Get leases 200"
time="2022-04-04T12:11:34.850Z" level=info msg="Update leases 200"
time="2022-04-04T12:11:39.863Z" level=info msg="Get leases 200"
time="2022-04-04T12:11:39.872Z" level=info msg="Update leases 200"
time="2022-04-04T12:11:40.162Z" level=info msg="List workflowtasksets 404"
E0404 12:11:40.163143       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.21.5/tools/cache/reflector.go:167: Failed to watch *v1alpha1.WorkflowTaskSet: failed to list *v1alpha1.WorkflowTaskSet: the server could not find the requested resource (get workflowtasksets.argoproj.io)
time="2022-04-04T12:11:44.888Z" level=info msg="Get leases 200"
time="2022-04-04T12:11:44.900Z" level=info msg="Update leases 200"
time="2022-04-04T12:11:49.910Z" level=info msg="Get leases 200"
time="2022-04-04T12:11:49.921Z" level=info msg="Update leases 200"
time="2022-04-04T12:11:52.936Z" level=info msg="List workflows 200"
time="2022-04-04T12:11:52.936Z" level=info msg=healthz age=5m0s err="<nil>" instanceID= labelSelector="!workflows.argoproj.io/phase,!workflows.argoproj.io/controller-instanceid" managedNamespace=
time="2022-04-04T12:11:54.936Z" level=info msg="Get leases 200"
time="2022-04-04T12:11:54.947Z" level=info msg="Update leases 200"
time="2022-04-04T12:11:59.958Z" level=info msg="Get leases 200"
time="2022-04-04T12:11:59.969Z" level=info msg="Update leases 200"
time="2022-04-04T12:12:04.983Z" level=info msg="Get leases 200"
time="2022-04-04T12:12:04.997Z" level=info msg="Update leases 200"
time="2022-04-04T12:12:10.009Z" level=info msg="Get leases 200"
time="2022-04-04T12:12:10.019Z" level=info msg="Update leases 200"
time="2022-04-04T12:12:15.035Z" level=info msg="Get leases 200"
time="2022-04-04T12:12:15.045Z" level=info msg="Update leases 200"
time="2022-04-04T12:12:20.055Z" level=info msg="Get leases 200"
time="2022-04-04T12:12:20.067Z" level=info msg="Update leases 200"
time="2022-04-04T12:12:25.087Z" level=info msg="Get leases 200"
time="2022-04-04T12:12:25.098Z" level=info msg="Update leases 200"
time="2022-04-04T12:12:27.887Z" level=info msg="List workflowtasksets 404"
E0404 12:12:27.887182       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.21.5/tools/cache/reflector.go:167: Failed to watch *v1alpha1.WorkflowTaskSet: failed to list *v1alpha1.WorkflowTaskSet: the server could not find the requested resource (get workflowtasksets.argoproj.io)
time="2022-04-04T12:12:30.110Z" level=info msg="Get leases 200"
time="2022-04-04T12:12:30.121Z" level=info msg="Update leases 200"
time="2022-04-04T12:12:35.135Z" level=info msg="Get leases 200"
time="2022-04-04T12:12:35.157Z" level=info msg="Update leases 200"
time="2022-04-04T12:12:40.168Z" level=info msg="Get leases 200"
time="2022-04-04T12:12:40.177Z" level=info msg="Update leases 200"
time="2022-04-04T12:12:45.188Z" level=info msg="Get leases 200"
time="2022-04-04T12:12:45.198Z" level=info msg="Update leases 200"
time="2022-04-04T12:12:47.504Z" level=info msg="Watch clusterworkflowtemplates 200"
time="2022-04-04T12:12:50.215Z" level=info msg="Get leases 200"
time="2022-04-04T12:12:50.226Z" level=info msg="Update leases 200"
time="2022-04-04T12:12:52.935Z" level=info msg="List workflows 200"
time="2022-04-04T12:12:52.935Z" level=info msg=healthz age=5m0s err="<nil>" instanceID= labelSelector="!workflows.argoproj.io/phase,!workflows.argoproj.io/controller-instanceid" managedNamespace=
time="2022-04-04T12:12:55.237Z" level=info msg="Get leases 200"
time="2022-04-04T12:12:55.246Z" level=info msg="Update leases 200"
time="2022-04-04T12:12:57.284Z" level=info msg="Alloc=7360 TotalAlloc=7316549 Sys=74065 NumGC=3515 Goroutines=202"
time="2022-04-04T12:13:00.259Z" level=info msg="Get leases 200"
time="2022-04-04T12:13:00.272Z" level=info msg="Update leases 200"
time="2022-04-04T12:13:05.282Z" level=info msg="Get leases 200"
time="2022-04-04T12:13:05.297Z" level=info msg="Update leases 200"
time="2022-04-04T12:13:10.308Z" level=info msg="Get leases 200"
time="2022-04-04T12:13:10.319Z" level=info msg="Update leases 200"
time="2022-04-04T12:13:11.675Z" level=info msg="Watch workflows 200"
time="2022-04-04T12:13:15.330Z" level=info msg="Get leases 200"
time="2022-04-04T12:13:15.339Z" level=info msg="Update leases 200"
time="2022-04-04T12:13:20.352Z" level=info msg="Get leases 200"
time="2022-04-04T12:13:20.362Z" level=info msg="Update leases 200"
time="2022-04-04T12:13:22.363Z" level=info msg="List workflowtasksets 404"
E0404 12:13:22.363688       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.21.5/tools/cache/reflector.go:167: Failed to watch *v1alpha1.WorkflowTaskSet: failed to list *v1alpha1.WorkflowTaskSet: the server could not find the requested resource (get workflowtasksets.argoproj.io)
time="2022-04-04T12:13:25.376Z" level=info msg="Get leases 200"
time="2022-04-04T12:13:25.388Z" level=info msg="Update leases 200"
time="2022-04-04T12:13:30.397Z" level=info msg="Get leases 200"
time="2022-04-04T12:13:30.406Z" level=info msg="Update leases 200"
time="2022-04-04T12:13:35.420Z" level=info msg="Get leases 200"
time="2022-04-04T12:13:35.430Z" level=info msg="Update leases 200"
time="2022-04-04T12:13:40.440Z" level=info msg="Get leases 200"
time="2022-04-04T12:13:40.451Z" level=info msg="Update leases 200"
time="2022-04-04T12:13:45.465Z" level=info msg="Get leases 200"
time="2022-04-04T12:13:45.475Z" level=info msg="Update leases 200"
time="2022-04-04T12:13:48.542Z" level=info msg="Watch workflows 200"
time="2022-04-04T12:13:50.485Z" level=info msg="Get leases 200"
time="2022-04-04T12:13:50.504Z" level=info msg="Update leases 200"
time="2022-04-04T12:13:52.938Z" level=info msg="List workflows 200"
time="2022-04-04T12:13:52.938Z" level=info msg=healthz age=5m0s err="<nil>" instanceID= labelSelector="!workflows.argoproj.io/phase,!workflows.argoproj.io/controller-instanceid" managedNamespace=
time="2022-04-04T12:13:55.523Z" level=info msg="Get leases 200"
time="2022-04-04T12:13:55.534Z" level=info msg="Update leases 200"
time="2022-04-04T12:13:58.420Z" level=info msg="Watch cronworkflows 200"
time="2022-04-04T12:14:00.544Z" level=info msg="Get leases 200"
time="2022-04-04T12:14:00.555Z" level=info msg="Update leases 200"
time="2022-04-04T12:14:02.377Z" level=info msg="Watch workflowtemplates 200"

# Logs from the workflow server:

time="2022-04-04T12:04:04.762Z" level=info msg="not enabling pprof debug endpoints"
time="2022-04-04T12:04:04.764Z" level=info authModes="[sso]" baseHRef=/ managedNamespace= namespace=argo-workflow secure=false
time="2022-04-04T12:04:04.764Z" level=warning msg="You are running in insecure mode. Learn how to enable transport layer security: https://argoproj.github.io/argo-workflows/tls/"
time="2022-04-04T12:04:04.764Z" level=info msg="config map" name=argo-workflow-gtw-argo-workflows-workflow-controller-configmap
time="2022-04-04T12:04:05.435Z" level=info msg="SSO configuration" clientId="{{argo-workflow-secret-infra-argo-workflow-oidc} client_id <nil>}" insecureSkipVerify=false issuer="****************" issuerAlias=DISABLED redirectUrl="***********************" scopes="[groups openid profile email openid]"
time="2022-04-04T12:04:05.537Z" level=info msg="SSO enabled"
time="2022-04-04T12:04:05.574Z" level=info msg="Starting Argo Server" instanceID= version=v3.3.1
time="2022-04-04T12:04:05.574Z" level=info msg="Creating DB session"
time="2022-04-04T12:04:05.792Z" level=info msg="Node status offloading config" ttl=5m0s
time="2022-04-04T12:04:05.792Z" level=info msg="Creating event controller" asyncDispatch=false operationQueueSize=16 workerCount=4
time="2022-04-04T12:04:05.808Z" level=info msg="GRPC Server Max Message Size, MaxGRPCMessageSize, is set" GRPC_MESSAGE_SIZE=104857600
time="2022-04-04T12:04:05.809Z" level=info msg="Argo Server started successfully on http://localhost:2746"
time="2022-04-04T12:04:32.976Z" level=info msg="selected SSO RBAC service account for user" email=***************** loginServiceAccount=argo-workflow-default-user-login serviceAccount=argo-workflow-default-user-login ssoDelegated=false ssoDelegationAllowed=false subject=c3235165-381d-4ddf-9a3f-e5d20f8dc4ac
time="2022-04-04T12:04:32.996Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=ListWorkflowTemplates grpc.service=workflowtemplate.WorkflowTemplateService grpc.start_time="2022-04-04T12:04:32Z" grpc.time_ms=24.762 span.kind=server system=grpc
time="2022-04-04T12:04:34.375Z" level=info msg="selected SSO RBAC service account for user" email=********************* loginServiceAccount=argo-workflow-default-user-login serviceAccount=argo-workflow-default-user-login ssoDelegated=false ssoDelegationAllowed=false subject=c3235165-381d-4ddf-9a3f-e5d20f8dc4ac
time="2022-04-04T12:04:34.387Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=GetWorkflowTemplate grpc.service=workflowtemplate.WorkflowTemplateService grpc.start_time="2022-04-04T12:04:34Z" grpc.time_ms=16.029 span.kind=server system=grpc
time="2022-04-04T12:04:39.266Z" level=info msg="selected SSO RBAC service account for user" email=******************* loginServiceAccount=argo-workflow-default-user-login serviceAccount=argo-workflow-default-user-login ssoDelegated=false ssoDelegationAllowed=false subject=c3235165-381d-4ddf-9a3f-e5d20f8dc4ac
time="2022-04-04T12:04:39.280Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=GetWorkflowTemplate grpc.service=workflowtemplate.WorkflowTemplateService grpc.start_time="2022-04-04T12:04:39Z" grpc.time_ms=18.811 span.kind=server system=grpc
time="2022-04-04T12:04:39.731Z" level=info msg="selected SSO RBAC service account for user" email=********************* loginServiceAccount=argo-workflow-default-user-login serviceAccount=argo-workflow-default-user-login ssoDelegated=false ssoDelegationAllowed=false subject=c3235165-381d-4ddf-9a3f-e5d20f8dc4ac
time="2022-04-04T12:04:39.732Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=GetVersion grpc.service=info.InfoService grpc.start_time="2022-04-04T12:04:39Z" grpc.time_ms=6.189 span.kind=server system=grpc
time="2022-04-04T12:04:45.469Z" level=info msg="selected SSO RBAC service account for user" email=******************** loginServiceAccount=argo-workflow-default-user-login serviceAccount=argo-workflow-default-user-login ssoDelegated=false ssoDelegationAllowed=false subject=c3235165-381d-4ddf-9a3f-e5d20f8dc4ac

# If the workflow's pods have not been created, you can skip the rest of the diagnostics.

# The workflow's pods that are problematic:
kubectl get pod -o yaml -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded

# Logs from in your workflow's wait container, something like:
kubectl logs -c wait -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.
@alexec
Copy link
Contributor

alexec commented Apr 4, 2022

Can you be more specific? For example, do you go into the UI, and are allowed to update a workflow?

What is your Kubernetes provider? Does it both support and have RBAC enabled correctly? E.g. Docker for Desktop does not support RBAC. Certain cloud configurations don't either.

@LoricAndre
Copy link
Contributor Author

Hi, thank you for that quick answer.

I can get into the UI without issues and the SSO SA is correctly assigned, which I can see in the User tab. The issue is that once logged in, I can create and submit workflows without any error.

My Kubernetes provider is Azure AKS, and RBAC is enabled and used successfully in other projects.

@alexec
Copy link
Contributor

alexec commented Apr 5, 2022 •
edited
Loading

I think this is most likely to be mis-configuration, so I'm don't want to invest too much time until we've checked that.

  • Can you confirm that the correct service account in being recieved by the Kubernetes API Server by checking your logs.
  • Can you double-check the service account using kubectl auth can-i?

If that fails, please book 30m via the new issue link.

@alexec
Copy link
Contributor

alexec commented Apr 7, 2022

Attempted to repro, failed:

argo-server | time="2022-04-07T14:38:12.194Z" level=info msg="selected SSO RBAC service account for user" email=kilgore@kilgore.trout loginServiceAccount=nothing serviceAccount=nothing ssoDelegated=false ssoDelegationAllowed=false subject=Cg0wLTM4NS0yODA4OS0wEgRtb2Nr
argo-server | time="2022-04-07T14:38:12.206Z" level=warning msg="finished unary call with code PermissionDenied" error="rpc error: code = PermissionDenied desc = workflows.argoproj.io is forbidden: User \"system:serviceaccount:argo:nothing\" cannot list resource \"workflows\" in API group \"argoproj.io\" in the namespace \"argo\"" grpc.code=PermissionDenied grpc.method=ListWorkflows grpc.service=workflow.WorkflowService grpc.start_time="2022-04-07T14:38:12-07:00" grpc.time_ms=13.813 span.kind=server system=grpc

@alexec
Copy link
Contributor

alexec commented Apr 7, 2022 

 kubectl auth can-i create workflows --as=system:serviceaccount:argo:nothing -n argo
no

@LoricAndre
Copy link
Contributor Author

LoricAndre commented Apr 12, 2022 •
edited by agilgur5
Loading

I think this is most likely to be mis-configuration, so I'm don't want to invest too much time until we've checked that.

  • Can you confirm that the correct service account in being recieved by the Kubernetes API Server by checking your logs.

It is, my email is associated with:
loginServiceAccount=argo-workflow-default-user-login serviceAccount=argo-workflow-default-user-login ssoDelegated=false ssoDelegationAllowed=false

  • Can you double-check the service account using kubectl auth can-i?

Impersonation is disabled on the cluster, I cannot test this.

@alexec alexec removed the triage label Sep 6, 2022
@qtheya
Copy link

qtheya commented Jul 13, 2023 

time="2023-07-13T09:08:42.134Z" level=info msg="selected SSO RBAC service account for user" email=****@****.** loginServiceAccount=tmp-sso-argo-workflows serviceAccount=tmp-sso-argo-workflows ssoDelegated=false ssoDelegationAllowed=false subject=**********
time="2023-07-13T09:08:42.135Z" level=info msg="selected SSO RBAC service account for user" email=****@****.** loginServiceAccount=tmp-sso-argo-workflows serviceAccount=tmp-sso-argo-workflows ssoDelegated=false ssoDelegationAllowed=false subject=**********
time="2023-07-13T09:08:42.139Z" level=info msg="tracking UI usage️️" email=****@****.**  name=openedSensorList subject=**********
time="2023-07-13T09:08:42.139Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=CollectEvent grpc.service=info.InfoService grpc.start_time="2023-07-13T09:08:42Z" grpc.time_ms=6.564 span.kind=server system=grpc

kubectl auth can-i list sensors --as=system:serviceaccount:argo:tmp-sso-argo-workflows -n argo
no

---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: argo:operator
rules:
  - apiGroups:
      - argoproj.io
    resources:
      - workflowtemplates
    resourceNames:
      - ci-k8s
      - ci-protobuf
      - ci-python
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - pods
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - pods/log
    verbs:
      - get
      - list
      - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: "tmp-sso-argo:operator"
subjects:
- kind: ServiceAccount
  name: tmp-sso-argo-workflows
  namespace: argo
roleRef:
  kind: Role
  name: argo:operator
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: "tmp-sso-argo-workflows"
  annotations:
    workflows.argoproj.io/rbac-rule: "'*****:****' in groups"
    workflows.argoproj.io/rbac-rule-precedence: "0"
secrets:
    - name: github-sso-argo-workflows

@umi0410
Copy link
Contributor

umi0410 commented Jul 20, 2023

@qtheya Does your comment(#8310 (comment)) mean that you succeeded to reproduce the bug?

@qtheya
Copy link

qtheya commented Jul 20, 2023

@qtheya Does your comment(#8310 (comment)) mean that you succeeded to reproduce the bug?

Yes

@gordonswing

This comment was marked as duplicate.

Sign in to view
@VLukyanov84

This comment was marked as duplicate.

Sign in to view
@tooptoop4
Copy link
Contributor

anyone can share the actual groups in userprofile and the actual 'something' in groups condition

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/sso-rbac type/bug type/security Security related
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@alexec @qtheya @gordonswing @umi0410 @tooptoop4 @LoricAndre @VLukyanov84