Waiting for ingress reached timeout

[kostiamol]

I'm deploying che in multi-user mode as a helm chart. During workspace loading I get this error:

On the basis of my previous experience I suppose that it's again a problem of my private cluster. But it's the second cluster I tried. Firstly it worked on the first cluster. But after some weeks I started to receive errors during che loading. So I decided to change cluster. And the same chart worked on the second cluster for 2 days. Now I'm experiencing this error. My admins have no idea what's going on. It's extremeley wierd. So my question is:
How can I find out what's going on? For now I see only 3 pods and their containers. Maybe I should check out workspace containers? But how?

[ghost]

@kostiamol when a workspace start, you should see a new namespace (this is the default behavior) in your k8s cluster.

Also, anything suspicious in server logs?

[kostiamol]

@eivantsov Yes, I have a new namespace:

Also I've checked the logs of che container in the che pod. But there were only [INFO] messages.

The thing is that something fails in the very beginning of workspace init because even Agent for command execution is not active.

[kostiamol]

@eivantsov Maybe the cause of the problem is the lack of rights? If so, how could I elevate them?

[ghost]

@kostiamol is this still an issue for you?

[kostiamol]

It's hard to say for sure but I think it was again a proxy issue.

[cq-z]

@kostiamol I also encountered the same problem. Do you have a solution?

[kostiamol]

@cq-z Nope ;(

[ghost]

@kostiamol do you see ingresses created? Anything suspicious in k8s events?

[kostiamol]

@eivantsov I didn't see anything suspicious. I guess it was a cluster internal error.

[ghost]

So, k8s wan't able to create ingresses.

What strategy did you use? Single host? Default host? Multi host? It's in helm chart

[cq-z]

Default host or Multi host Neither！

[ghost]

@cq-z do you see created ingresses?

[cq-z]

Have！

[ghost]

@cq-z i cannot read Chinese but it looks like ingresses did not receive their endpoints.

[cq-z]

2018-11-26 10:56:03,479[io-8080-exec-10]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 322]   - Starting workspace 'admin/wksp-7crm' with id 'workspacend2cqkoke5xfsjy5' by user 'admin'
2018-11-26 10:56:04,078[aceSharedPool-0]  [WARN ] [i.f.k.c.i.VersionUsageUtils 55]      - The client is using resource type 'ingresses' with unstable version 'v1beta1'
2018-11-26 11:01:04,270[aceSharedPool-0]  [WARN ] [.i.k.KubernetesInternalRuntime 228]  - Failed to start Kubernetes runtime of workspace workspacend2cqkoke5xfsjy5. Cause: Waiting for ingress 'ingressnf1ac8jz' reached timeout
2018-11-26 11:01:05,076[aceSharedPool-0]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 383]   - Workspace 'admin:wksp-7crm' with id 'workspacend2cqkoke5xfsjy5' start failed
2018-11-26 11:03:51,085[io-8080-exec-12]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 322]   - Starting workspace 'admin/wksp-7crm' with id 'workspacend2cqkoke5xfsjy5' by user 'admin'
2018-11-26 11:08:51,347[aceSharedPool-1]  [WARN ] [.i.k.KubernetesInternalRuntime 228]  - Failed to start Kubernetes runtime of workspace workspacend2cqkoke5xfsjy5. Cause: Waiting for ingress 'ingress036ybeia' reached timeout
2018-11-26 11:08:51,533[aceSharedPool-1]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 383]   - Workspace 'admin:wksp-7crm' with id 'workspacend2cqkoke5xfsjy5' start failed

[cq-z]

ingress036ybeia has been created but is invalid

[cq-z]

hello @eivantsov Is there any solution?

[xuchenhao001]

Got the same issue, when I create a namespace, there alway errors like this:

Error: Failed to run the workspace: "Waiting for ingress 'ingressh0t7poya' reached timeout"

I have checked that, there are 4 ingresses, 1 service and 1 pvc successfully created under a new namespace when creating new workspace.
However, there are no error message in che-server pod.

[cq-z]

@xuchenhao001 I AM switch to helm reload ingress . it's OK ！

[xuchenhao001]

@cq-z Congratulations! So this bug is because of the nodePort ingress? Could you please give me some links/docs about helm reload ingress? thanks!

[cq-z]

helm  upgrade --install nginx-ingress  --namespace nginx-ingress -f values.yaml stable/nginx-ingress

https://github.com/helm/charts/tree/master/stable/nginx-ingress
https://docs.helm.sh/using_helm/