Unable to load workflow

[gupash]

Is this a BUG REPORT or FEATURE REQUEST?: Help Wanted

What happened: Argo UI unable to load workflow

What you expected to happen: Argo UI to work properly

Anything else we need to know?: Argo is installed in a namespace

Environment:

• Argo version: 2.1.1
• Kubernetes version : 1.10

Other debugging information (if applicable):

When created a new argo workflow, found the below exception in workflow-controller logs:

time="2018-08-23T18:34:58Z" level=info msg="Failed to create pod hello-world-zctrk (hello-world-zctrk): admission webhook \"opa.cloud.io\" denied the request: hostpath cannot be used by unprivileged users, nor by privileged users with non-whitelisted paths" namespace=2304613691 workflow=hello-world-zctrk
time="2018-08-23T18:34:58Z" level=info msg="Pod node hello-world-zctrk (hello-world-zctrk) initialized Error (message: admission webhook \"opa.cloud.io\" denied the request: hostpath cannot be used by unprivileged users, nor by privileged users with non-whitelisted paths)" namespace=2304613691 workflow=hello-world-zctrk
time="2018-08-23T18:34:58Z" level=info msg="Updated phase Running -> Error" namespace=2304613691 workflow=hello-world-zctrk
time="2018-08-23T18:34:58Z" level=info msg="Updated message  -> admission webhook \"opa.cloud.io\" denied the request: hostpath cannot be used by unprivileged users, nor by privileged users with non-whitelisted paths" namespace=2304613691 workflow=hello-world-zctrk

[jessesuen]

What about argo list ?

[jessesuen]

Also, if you open developer console to show what the error is, that would help. Also the tail of the argo-ui pod.

[gupash]

Returns blank.
argo list
NAME STATUS AGE DURATION

[gupash]

argo UI pod doesn't produce any logs. How do you enable the developer console ?

[gupash]

$kubectl logs argo-ui-964b7bc84-5w8rz -f
<blank>

[gupash]

$argo submit argo-hello-world.yaml
Name:                hello-world-zctrk
Namespace:           2304613691
ServiceAccount:      default
Status:              Pending
Created:             Thu Aug 23 11:34:58 -0700 (now)


$kubectl get pods -w
NAME                                 READY     STATUS    RESTARTS   AGE
argo-ui-964b7bc84-5w8rz              1/1       Running   0          13m
workflow-controller-7846cf48-9fn7x   1/1       Running   0          14m


$argo list
NAME                STATUS   AGE    DURATION
hello-world-zctrk   Error    18s    0s


$kubectl logs argo-ui-964b7bc84-5w8rz

$kubectl logs workflow-controller-7846cf48-9fn7x
time="2018-08-23T18:20:17Z" level=info msg="workflow controller configuration from workflow-controller-configmap:\nartifactRepository: {}\nnamespace: 2304613691\nexecutorImage: argoexec:v2.1.1\n"
time="2018-08-23T18:20:17Z" level=info msg="Workflow Controller (version: v2.1.1) starting"
time="2018-08-23T18:20:17Z" level=info msg="Watch Workflow controller config map updates"
time="2018-08-23T18:20:17Z" level=info msg="Detected ConfigMap update. Updating the controller config."
time="2018-08-23T18:20:17Z" level=info msg="workflow controller configuration from workflow-controller-configmap:\nartifactRepository: {}\nnamespace: 2304613691\nexecutorImage: argoexec:v2.1.1\n"
time="2018-08-23T18:25:17Z" level=info msg="Alloc=2733 TotalAlloc=28352 Sys=14282 NumGC=14 Goroutines=50"
W0823 18:26:29.069165       1 reflector.go:341] github.com/argoproj/argo/workflow/controller/controller.go:140: watch of *v1.Pod ended with: too old resource version: 122240112 (122247300)
time="2018-08-23T18:30:17Z" level=info msg="Alloc=2770 TotalAlloc=28748 Sys=14282 NumGC=16 Goroutines=50"
time="2018-08-23T18:34:58Z" level=info msg="Processing workflow" namespace=2304613691 workflow=hello-world-zctrk
time="2018-08-23T18:34:58Z" level=info msg="Updated phase  -> Running" namespace=2304613691 workflow=hello-world-zctrk
time="2018-08-23T18:34:58Z" level=info msg="Failed to create pod hello-world-zctrk (hello-world-zctrk): admission webhook \"opa.cloud.io\" denied the request: hostpath cannot be used by unprivileged users, nor by privileged users with non-whitelisted paths" namespace=2304613691 workflow=hello-world-zctrk
time="2018-08-23T18:34:58Z" level=info msg="Pod node hello-world-zctrk (hello-world-zctrk) initialized Error (message: admission webhook \"opa.cloud.io\" denied the request: hostpath cannot be used by unprivileged users, nor by privileged users with non-whitelisted paths)" namespace=2304613691 workflow=hello-world-zctrk
time="2018-08-23T18:34:58Z" level=info msg="Updated phase Running -> Error" namespace=2304613691 workflow=hello-world-zctrk
time="2018-08-23T18:34:58Z" level=info msg="Updated message  -> admission webhook \"opa.cloud.io\" denied the request: hostpath cannot be used by unprivileged users, nor by privileged users with non-whitelisted paths" namespace=2304613691 workflow=hello-world-zctrk
time="2018-08-23T18:34:58Z" level=info msg="Marking workflow completed" namespace=2304613691 workflow=hello-world-zctrk
time="2018-08-23T18:34:58Z" level=info msg="Workflow update successful" namespace=2304613691 workflow=hello-world-zctrk
time="2018-08-23T18:35:17Z" level=info msg="Alloc=5019 TotalAlloc=41729 Sys=15594 NumGC=23 Goroutines=50"

[jessesuen]

admission webhook "opa.applecloud.io" denied the request: hostpath cannot be used by unprivileged users, nor by privileged users with non-whitelisted paths"

You are hitting #942

This will be addressed with this feature: #902 currently in review.

[gupash]

Is there any timeline when #902 might make it to new version.

Also If I want to make it work now and somehow talk my cluster admin's into allowing the access,
by creating a cluster-role and cluster-binding which in turn we can refer in role and role-binding in our own namespace, what are the minimum resources and privileges, it can work with.

[jessesuen]

Is there any timeline when #902 might make it to new version.

The review is pretty much completed with some minor changes left. So it should make v2.2 which we're targetting now for next week.

Also If I want to make it work now and somehow talk my cluster admin's into allowing the access,
by creating a cluster-role and cluster-binding which in turn we can refer in role and role-binding in our own namespace, what are the minimum resources and privileges, it can work with.

It may or may not help, since it depends on how your admission controller (OPA) is set up. Your admission controller is rejecting pods which want to mount host paths, which makes sense from a security standpoint. If OPA takes into consideration the service account which the pods run as, to decide if this is okay, you may get this to work by doing:

argo submit --serviceaccount <someprivilegedserviceaccount>

But again, I think it's dependent on your environment.

[geniekanth]

We are tracking this issue. Based on your comment, I tried it on v2.2 and I am getting the same error. Could you please explain on why we need host path details?

[alexmt]

Hello @geniekanth . The issue was finally fixed in master. Please check #1114 (comment) if you need workaround.