Support the semantic highlighting based on the LSP extension proposal

[kittaakos]

I have submitted a proposal to extend the LSP with the semantic highlighting support.

I have already adjusted the LSP4J API based on my proposal. See also the vscode-languageserver-node implementation

Plans:

• Adopt the Java LS to support the semantic highlightings. My plan is to copy over a bunch of org.eclipse.jdt.internal.ui.javaeditor.Semantic* classes from the JDT UI to the LS and get rid of the Eclipse Workbench, JFace, and SWT dependencies.
• The goal is to support the LS based semantic highlighting in Eclipse Theia, VS Code, and Eclipse Che.

Suggestions and feedback are welcome. Thanks!

Edit: updated the plans.

[fbricon]

@kittaakos be aware the Technology PMC is currently frowning very hard at jdt.ls forking jdt.ui classes over (see #684 (comment)). They'd strongly prefer to refactor jdt.ui code over to jdt.core, if doable.

[kittaakos]

Thank you for the note, @fbricon.

CC: @svenefftinge

[svenefftinge]

@kittaakos be aware the Technology PMC is currently frowning very hard at jdt.ls forking jdt.ui classes over (see #684 (comment)). They'd strongly prefer to refactor jdt.ui code over to jdt.core, if doable.

So you think we should

1. extract the code needed to from jdt.ui to jdt.core
2. make use of those changes in jdt.ls

Sounds more sustainable but also like a lot of extra rounds :|

[fbricon]

Sounds more sustainable but also like a lot of extra rounds :|

Tell me about it.
But we have @rgrunber and @jjohnstn to help if needed

[svenefftinge]

We could go in parallel. I.e.

• do a PR here with the copied code
• do a PR for jdt.core moving the code.

And then once jdt.ls upgrades to a version that has those changes we could remove the copied code here.

[rgrunber]

For contributing back to JDT, given that the code you're copying comes from org.eclipse.jdt.ui, it would be easier to move it into org.eclipse.jdt.core.manipulation (resides in same git repo as org.eclipse.jdt.ui) as opposed to org.eclipse.jdt.core, which is a separate repository.

When copied into JDT-LS much of the functionality needed for the JDT UI could be removed but would need to be preserved if contributing upstream.

You could then just register an account at Eclipse and push your changes for review to ssh://username@git.eclipse.org:29418/jdt/eclipse.jdt.ui

[kittaakos]

The following are the JDT UI dependencies I am currently resuing for this task:

• https://github.com/eclipse/eclipse.jdt.ui/blob/d41fa3326c5b75a6419c81fcecb37d7d7fb3ac43/org.eclipse.jdt.ui/ui/org/eclipse/jdt/internal/ui/javaeditor/SemanticHighlightings.java#L58-L1653
  • These are without any SWT/Preferences dependencies. We need only the boolean consumes(SemanticToken) and the boolean consumesLiteral(SemanticToken) implementations.
• https://github.com/eclipse/eclipse.jdt.ui/blob/d41fa3326c5b75a6419c81fcecb37d7d7fb3ac43/org.eclipse.jdt.ui/ui/org/eclipse/jdt/internal/ui/javaeditor/SemanticHighlightingReconciler.java#L66-L216

We do not need:

• The entire Eclipse Job support built around the semantic highlighting capability in JDT UI.
• We do not need want the mutable nature of the Position instances plus the position updater, and semantic highlighting reconciliation.

[kittaakos]

A work in progress PR for the task: #746

[kittaakos]

@rgrunber, @fbricon please take a look and give feedback. Thank you!