Java plugin not working in a Che 7 workspace with 2 other plugins on che.openshift.io (not respecting memory limit) #13146

apupier · 2019-04-15T09:17:30Z

Description

When i try to activate the language Support by Java on a Che 7 workspace in che.openshift.io, there is a message stating that java server crashed 5 times and won't be restarted.

Reproduction Steps

go to che.openshift.io
create a Che 7workspace
modify workspace to include Language Support for java workspace
restart workspce
wait a bit...
--> check in Output -> language SSupport for java, there is

[Info  - 09:11:38] Connection to server got closed. Server will restart.
[Info  - 09:11:52] Connection to server got closed. Server will restart.
[Info  - 09:12:15] Connection to server got closed. Server will restart.
[Info  - 09:12:32] Connection to server got closed. Server will restart.
[Error - 09:12:46] Connection to server got closed. Server will not be restarted.

OS and version:

Diagnostics:

The text was updated successfully, but these errors were encountered:

l0rd · 2019-04-15T11:58:12Z

@apupier you need to set memory as it's done here.

That should be solved once the new Che 7 stacks will be available on che.openshift.io (cc @ibuziuk).

apupier · 2019-04-15T12:42:58Z

Unfortunately, it is not possible to apply the workaround by default on che.openshift.io:

Error: Failed to run the workspace: "Unrecoverable event occurred: 'FailedCreate', 'Error creating: pods "workspacelquowze86j7hfa5x.ws-56dd8dbc5c-fks26" is forbidden: [maximum cpu usage per Pod is 6, but limit is 7666m., maximum memory usage per Pod is 3Gi, but limit is 4117594112.]', 'workspacelquowze86j7hfa5x.ws-56dd8dbc5c'"

apupier · 2019-04-15T12:46:19Z

I tried to increase the workspace RAm to 2G but it then fails with similar error:

Error: Failed to run the workspace: "Unrecoverable event occurred: 'FailedCreate', 'Error creating: pods "workspacelquowze86j7hfa5x.ws-7844dd65ff-hwn7h" is forbidden: [maximum cpu usage per Pod is 6, but limit is 10666m., maximum memory usage per Pod is 3Gi, but limit is 5728206848.]', 'workspacelquowze86j7hfa5x.ws-7844dd65ff'"

apupier · 2019-04-15T12:51:43Z

tried also with 3G which the max allowed per workspace, still same error

l0rd · 2019-04-15T13:23:55Z

You need to set Theia memory as well:

  "attributes": {
    "plugins": "che-machine-exec-plugin:0.0.1,org.eclipse.che.vscode-redhat.java:0.38.0",
    "editor": "org.eclipse.che.editor.theia:1.0.0",
    "sidecar.org.eclipse.che.vscode-redhat.java.memory_limit": "1500Mi",
    "sidecar.org.eclipse.che.editor.theia.memory_limit": "512Mi"
  },

UPDATE using version next of Theia doesn't work currently. I have edited this comment to use version 1.0.0 instead

apupier · 2019-04-15T13:55:37Z

I tried with it, i still have the same error:

Cleaning /plugins dir
Starting Init Plugin Broker
Started Che Plugin Broker
List of plugins and editors to install
- che-machine-exec-plugin:0.0.1 - Che Plug-in with che-machine-exec service to provide creation terminal or tasks for Eclipse CHE workspace machines.
- org.eclipse.che.editor.theia:1.0.0 - Eclipse Theia
Starting common Che plugins processing
All plugins have been successfully processed
Started VS Code Plugin Broker
Starting VS Code extensions processing
List of plugins and editors to install
- org.eclipse.che.vscode-redhat.java:0.38.0 - Java Linting, Intellisense, formatting, refactoring, Maven/Gradle support and more...
All plugins have been successfully processed
Error: Failed to run the workspace: "Unrecoverable event occurred: 'FailedCreate', 'Error creating: pods "workspacelquowze86j7hfa5x.ws-59f65fb58f-tdqcp" is forbidden: [maximum memory usage per Pod is 3Gi, but limit is 5488246784., maximum cpu usage per Pod is 6, but limit is 10221m.]', 'workspacelquowze86j7hfa5x.ws-59f65fb58f'"

apupier · 2019-04-15T14:00:29Z

ah I need to have the 2 memory limits set and to not have the workspace RAM set too high (I switched back to 0.5G and it is working)

thanks for your help!

l0rd · 2019-04-15T14:14:00Z

@apupier ~~I think you are facing https://github.com/redhat-developer/che-functional-tests/issues/473~~ sorry I didn't read your last message. Ok so yes you need to do some math: 500 Theia + 1000 Java + 500 runtime container + 250 JWT...you still have 750 MB for something else.

l0rd · 2019-04-15T14:18:56Z

Eventually this should be done automatically after #11801 get fixed

ibuziuk · 2019-04-15T14:49:27Z

@apupier as I understand you was able to run workspace eventually, right?

apupier-demo-che 1/1 Running 0 5m jwt-proxy
apupier-demo-che 6/6 Running 0 5m workspace pod

apupier · 2019-04-15T15:35:32Z

the workspace is starting.
I was able to have java working if I have only it (at least no error message but i must admit that i have not tested any completion)

if I have the 3 plugins, the workspace is starting but java language server is not working, I have the error [Error - 14:14:02] Connection to server got closed. Server will not be restarted. error

apupier · 2019-04-16T13:01:02Z

I was able to access the pod log, seems nothing interesting:

Plugins Endpoints are  [ 'ws://plqazfooow:6795',
--
  | 'ws://rkzvynozdc:9124',
  | 'ws://pdgejzuioe:7598' ]
  | Plugins Mapping Endpoints are  Map {
  | 'redhat_vscode_xml' => 'ws://plqazfooow:6795',
  | 'redhat_java' => 'ws://rkzvynozdc:9124',
  | 'camel_tooling_vscode_apache_camel' => 'ws://pdgejzuioe:7598' }
  | Starting the deployer with the list of resolvers [ LocalDirectoryPluginDeployerResolver {},
  | GithubPluginDeployerResolver { unpackedFolder: '/tmp/github-remote' },
  | HttpPluginDeployerResolver { unpackedFolder: '/tmp/http-remote' },
  | VsCodePluginDeployerResolver { unpackedFolder: '/tmp/vscode-extension-marketplace' } ]
  | Theia Endpoint 19/pid listening on port 9124
  | Found the list of default plugins ID on env: undefined
  | Found the list of plugins ID on env: undefined
  | Found the list of default plugins ID from CLI: undefined
  | PLUGIN_HOST(19): PluginManagerExtImpl/init()
  | PLUGIN_HOST(19): initializing(/home/theia/node_modules/@theia/plugin-ext-vscode/lib/node/plugin-vscode-init.js)
  | PLUGIN_HOST(19): PluginManagerExtImpl/loadPlugin(/plugins/org.eclipse.che.vscode-redhat.java.0.38.0/extension/dist/extension)

apupier · 2019-04-16T13:16:32Z

the sidecar pod containing the Java LS is marked to have 150MiB available only:

although I mentioned "sidecar.org.eclipse.che.camel-tooling.vscode-redhat.java.memory_limit": "1280Mi"

Shouldn't be indicated 1280instead of 150 at this place?

l0rd · 2019-04-16T17:06:27Z

@apupier I think this may be a bug. Can you share your attributes section so that we can try to reproduce it?

apupier · 2019-04-17T06:53:42Z

Can you share your attributes section so that we can try to reproduce it?

here is the latest that I tried:

"attributes": {
    "editor": "org.eclipse.che.editor.theia:1.0.0",
    "sidecar.org.eclipse.che.redhat.vscode-xml.memory_limit": "128Mi",
    "sidecar.org.eclipse.che.camel-tooling.vscode-redhat.java.memory_limit": "1280Mi",
    "plugins": "che-machine-exec-plugin:0.0.1,https://raw.githubusercontent.com/apupier/che-plugin-registry/109-AddApacheCamelVSCodeExtension/plugins/camel-tooling.vscode-apache-camel:0.0.14,org.eclipse.che.vscode-redhat.java:0.38.0,redhat.vscode-xml:0.3.0",
    "sidecar.org.eclipse.che.editor.theia.memory_limit": "512Mi",
    "sidecar.org.eclipse.che.camel-tooling.vscode-apache-camel.memory_limit": "128Mi"
  }

apupier · 2019-04-17T12:19:24Z

OMG, rereading, the alue is wrong:
"sidecar.org.eclipse.che.camel-tooling.vscode-redhat.java.memory_limit": "1280Mi",
shoudl be
"sidecar.org.eclipse.che.vscode-redhat.java.memory_limit": "1280Mi",

apupier · 2019-04-17T12:28:55Z

it is not working either:

 "attributes": {
    "editor": "org.eclipse.che.editor.theia:1.0.0",
    "sidecar.org.eclipse.che.redhat.vscode-xml.memory_limit": "128Mi",
    "sidecar.org.eclipse.che.vscode-redhat.java.memory_limit": "1280Mi",
    "plugins": "che-machine-exec-plugin:0.0.1,https://raw.githubusercontent.com/apupier/che-plugin-registry/109-AddApacheCamelVSCodeExtension/plugins/camel-tooling.vscode-apache-camel:0.0.14,org.eclipse.che.vscode-redhat.java:0.38.0,redhat.vscode-xml:0.3.0",
    "sidecar.org.eclipse.che.editor.theia.memory_limit": "512Mi",
    "sidecar.org.eclipse.che.camel-tooling.vscode-apache-camel.memory_limit": "128Mi"
  }

l0rd · 2019-04-17T12:51:28Z

The same error? Because we have redhat-developer/rh-che#1371 as well on che.openshift.io

apupier · 2019-04-17T13:24:13Z

with the lastly provided attributes config, I had no error at all. Just no completion or hover or anything that seems related to Java. At the bottom it was blocked on "57% Java initialization" (or something like that)

In case, I increase the memory limits for the extension, I have the same kind of error pointed by redhat-developer/rh-che#1371

EDIT: realizing the error message is not exactly the same. It is a memory error that i have, not a service limit error.

ibuziuk · 2019-04-17T18:47:54Z

also can repro quota issue with a few plugins enabled:

pulling image "eclipse/che-remote-plugin-runner-java8:next"
Error: Failed to run the workspace: "Unrecoverable event occurred: 'FailedCreate', 'Error creating: pods "workspaceo45z391nqghagzfs.che-jwtproxy-647c78c7df-s2dtw" is forbidden: exceeded quota: compute-resources, requested: limits.cpu=292m,limits.memory=157286400, used: limits.cpu=5905m,limits.memory=3173875712, limited: limits.cpu=6,limits.memory=3Gi', 'workspaceo45z391nqghagzfs.che-jwtproxy-647c78c7df'"

tsmaeder · 2019-04-26T12:19:25Z

@apupier @l0rd @ibuziuk I don't really think this is a bug: if the limits on openshift.io are less than what we need, we can either not support this use case on openshift.io, increase the limits or create a special configuration that runs within those limitations.

What I am really saying is: what would be an action that would solve this case?

apupier · 2019-04-26T12:29:59Z

I think there are 2 slightly different issues:

one is that, by default, when using several extensions, the memory available on che.openshift.io are not enough. I think it is the case that "the limits on openshift.io are less than what we need, we can either not support this use case on openshift.io, increase the limits or create a special configuration that runs within those limitations."
the second is that even after setting memory limits for each sidecars, it is still not working as it seems that the memory limits are not respected. see Java plugin not working in a Che 7 workspace with 2 other plugins on che.openshift.io (not respecting memory limit) #13146 (comment)

l0rd · 2019-04-26T17:42:45Z

@tsmaeder you are right for memory limits (there is not much we can do) but for the services limit we are creating more than 15 services and most of them are only for inter-container communications (i.e. we do not really need 15 services) hence there is still room for improvement.

@apupier isn't your second issue related to redhat-developer/rh-che#1371? In this case we may fix that when we fix #11018 (@davidfestal task for next sprint)

apupier · 2019-04-29T06:33:33Z

@apupier isn't your second issue related to redhat-developer/rh-che#1371?

I don't think that it is the same issue. In redhat-developer/rh-che#1371 , it is a problem of number of services. here, i'm experiencing memory limit issues which seems to not be respected.

apupier · 2019-05-14T08:23:36Z

I tried again with theia:next and the camel extension which is now in official registry and i still hit the same problem. Here is the config attributes:

  "attributes": {
    "editor": "org.eclipse.che.editor.theia:next",
    "sidecar.org.eclipse.che.redhat.vscode-xml.memory_limit": "128Mi",
    "sidecar.org.eclipse.che.vscode-redhat.java.memory_limit": "1280Mi",
    "plugins": "che-machine-exec-plugin:0.0.1,org.eclipse.che.vscode-redhat.java:0.38.0,camel-tooling.vscode-apache-camel:0.0.14,redhat.vscode-xml:0.5.1",
    "sidecar.org.eclipse.che.editor.theia.memory_limit": "512Mi",
    "sidecar.org.eclipse.che.camel-tooling.vscode-apache-camel.memory_limit": "128Mi"
  }

tsmaeder · 2019-05-14T09:35:24Z

@apupier with the recent changes in the plugin registry, you probably need to update the attributes to the new format of plugin id's: "sidecar.redhat/java.memory_limit": "1500Mi"

apupier · 2019-05-14T09:57:13Z

I tried with

 "attributes": {
    "editor": "org.eclipse.che.editor.theia:next",
    "sidecar.eclipse/che-theia.memory_limit": "512Mi",
    "sidecar.org.eclipse.che.vscode-redhat.java.memory_limit": "1280Mi",
    "plugins": "che-machine-exec-plugin:0.0.1,org.eclipse.che.vscode-redhat.java:0.43.0,camel-tooling.vscode-apache-camel:0.0.14,redhat.vscode-xml:0.5.1",
    "sidecar.redhat/java.memory_limit": "1500Mi",
    "sidecar.camel-tooling/vscode-apache-camel.memory_limit": "128Mi",
    "sidecar.org.eclipse.che.editor.theia.memory_limit": "512Mi",
    "sidecar.org.eclipse.che.redhat.vscode-xml.memory_limit": "128Mi",
    "sidecar.redhat/vscode-xml.memory_limit": "128Mi"
  },

the sidecar seems to use the 1280M that I assigned to it (so means it is still using the old notation)

and i finally have both 3 plugins working altogether!!! (after waiting for several mintutes that Java Language Server is ready)
Java completion:

Camel URI completion for Java:

Camel URI completion for XML:

XML completion:

apupier · 2019-05-14T13:23:26Z

closing as it is working now by setting memory limits manually (even if the availability of the Java Language Server is taking more than 10 minutes)

apupier mentioned this issue Apr 16, 2019

Camel Plugin for Che 7 #12584

Closed

23 tasks

This was referenced Apr 23, 2019

Provide Camel stack for Che 7 #13182

Closed

Add Camel WSDl2Rest VS Code Extension (#120) eclipse-che/che-plugin-registry#124

Closed

apupier changed the title ~~Java plugin not working in a Che 7 workspace on che.openshift.io~~ Java plugin not working in a Che 7 workspace with 2 other plugins on che.openshift.io (not respecting memory limit) May 14, 2019

apupier closed this as completed May 15, 2019

apupier mentioned this issue Jul 23, 2019

[Regression] Apache Camel devfile not starting on Hosted Che #13832

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Java plugin not working in a Che 7 workspace with 2 other plugins on che.openshift.io (not respecting memory limit) #13146

Java plugin not working in a Che 7 workspace with 2 other plugins on che.openshift.io (not respecting memory limit) #13146

apupier commented Apr 15, 2019

l0rd commented Apr 15, 2019

apupier commented Apr 15, 2019

apupier commented Apr 15, 2019

apupier commented Apr 15, 2019

l0rd commented Apr 15, 2019 •

edited

Loading

apupier commented Apr 15, 2019

apupier commented Apr 15, 2019

l0rd commented Apr 15, 2019 •

edited

Loading

l0rd commented Apr 15, 2019

ibuziuk commented Apr 15, 2019

apupier commented Apr 15, 2019 •

edited

Loading

apupier commented Apr 16, 2019

apupier commented Apr 16, 2019 •

edited

Loading

l0rd commented Apr 16, 2019

apupier commented Apr 17, 2019

apupier commented Apr 17, 2019

apupier commented Apr 17, 2019

l0rd commented Apr 17, 2019

apupier commented Apr 17, 2019 •

edited

Loading

ibuziuk commented Apr 17, 2019

tsmaeder commented Apr 26, 2019

apupier commented Apr 26, 2019

l0rd commented Apr 26, 2019

apupier commented Apr 29, 2019 •

edited

Loading

apupier commented May 14, 2019 •

edited

Loading

tsmaeder commented May 14, 2019

apupier commented May 14, 2019 •

edited

Loading

apupier commented May 14, 2019 •

edited

Loading

Java plugin not working in a Che 7 workspace with 2 other plugins on che.openshift.io (not respecting memory limit) #13146

Java plugin not working in a Che 7 workspace with 2 other plugins on che.openshift.io (not respecting memory limit) #13146

Comments

apupier commented Apr 15, 2019

Description

Reproduction Steps

l0rd commented Apr 15, 2019

apupier commented Apr 15, 2019

apupier commented Apr 15, 2019

apupier commented Apr 15, 2019

l0rd commented Apr 15, 2019 • edited Loading

apupier commented Apr 15, 2019

apupier commented Apr 15, 2019

l0rd commented Apr 15, 2019 • edited Loading

l0rd commented Apr 15, 2019

ibuziuk commented Apr 15, 2019

apupier commented Apr 15, 2019 • edited Loading

apupier commented Apr 16, 2019

apupier commented Apr 16, 2019 • edited Loading

l0rd commented Apr 16, 2019

apupier commented Apr 17, 2019

apupier commented Apr 17, 2019

apupier commented Apr 17, 2019

l0rd commented Apr 17, 2019

apupier commented Apr 17, 2019 • edited Loading

ibuziuk commented Apr 17, 2019

tsmaeder commented Apr 26, 2019

apupier commented Apr 26, 2019

l0rd commented Apr 26, 2019

apupier commented Apr 29, 2019 • edited Loading

apupier commented May 14, 2019 • edited Loading

tsmaeder commented May 14, 2019

apupier commented May 14, 2019 • edited Loading

apupier commented May 14, 2019 • edited Loading

l0rd commented Apr 15, 2019 •

edited

Loading

l0rd commented Apr 15, 2019 •

edited

Loading

apupier commented Apr 15, 2019 •

edited

Loading

apupier commented Apr 16, 2019 •

edited

Loading

apupier commented Apr 17, 2019 •

edited

Loading

apupier commented Apr 29, 2019 •

edited

Loading

apupier commented May 14, 2019 •

edited

Loading

apupier commented May 14, 2019 •

edited

Loading

apupier commented May 14, 2019 •

edited

Loading