Add experimental Semantic Highlighter #3667

JoeRobich · 2020-03-15T00:25:24Z

Implementation of the Semantic Tokens provider (microsoft/vscode#86415) requested in #3565. Built on the Semantic Endpoint added to OmniSharp in OmniSharp/omnisharp-roslyn#1734

Future enhancement:

Add RegEx classifications

With Textmate Classification:

With Semantic Classification:

With Semantic Classification and Visual Studio 2019 theme tweaks:

Applied theme tweaks:

    "editor.tokenColorCustomizations": {
        "[Default Dark+]": {
            "textMateRules": [
                {
                    "scope": "entity.name.namespace",
                    "settings": {
                        "foreground": "#D4D4D4"
                    }
                },
                {
                    "scope": "entity.name.variable.field",
                    "settings": {
                        "foreground": "#D4D4D4"
                    }
                },
                {
                    "scope": "variable.other.property",
                    "settings": {
                        "foreground": "#D4D4D4"
                    }
                },
                {
                    "scope": "variable.other.constant",
                    "settings": {
                        "foreground": "#D4D4D4",
                    }
                },
                {
                    "scope": "variable.other.enummember",
                    "settings": {
                        "foreground": "#D4D4D4"
                    }
                },
                {
                    "scope": "entity.name.type.interface",
                    "settings": {
                        "foreground": "#b8d7a3"
                    }
                },
                {
                    "scope": "entity.name.type.enum",
                    "settings": {
                        "foreground": "#b8d7a3"
                    }
                },
                {
                    "scope": "entity.name.type.parameter",
                    "settings": {
                        "foreground": "#b8d7a3"
                    }
                },
                {
                    "scope": "entity.name.type.struct",
                    "settings": {
                        "foreground": "#86C691",
                    }
                },
                {
                    "scope": "entity.name.function.extension",
                    "settings": {
                        "foreground": "#DCDCAA",
                    }
                },
                {
                    "scope": "comment.documentation",
                    "settings": {
                        "foreground": "#608B4E",
                    }
                },
                {
                    "scope": "comment.documentation.attribute",
                    "settings": {
                        "foreground": "#C8C8C8",
                    }
                },
                {
                    "scope": "comment.documentation.cdata",
                    "settings": {
                        "foreground": "#E9D585",
                    }
                },
                {
                    "scope": "comment.documentation.delimiter",
                    "settings": {
                        "foreground": "#808080",
                    }
                },
                {
                    "scope": "comment.documentation.name",
                    "settings": {
                        "foreground": "#569CD6",
                    }
                },
                {
                    "scope": "comment.regex",
                    "settings": {
                        "foreground": "#57A64A",
                    }
                },
                {
                    "scope": "constant.character.character-class.regexp",
                    "settings": {
                        "foreground": "#2EABFE",
                    }
                },
                {
                    "scope": "keyword.control.anchor.regexp",
                    "settings": {
                        "foreground": "#F979AE",
                    }
                },
                {
                    "scope": "keyword.operator.quantifier.regexp",
                    "settings": {
                        "foreground": "#F979AE",
                    }
                },
                {
                    "scope": "punctuation.definition.group.regexp",
                    "settings": {
                        "foreground": "#05C3BA",
                    }
                },
                {
                    "scope": "keyword.operator.or.regexp",
                    "settings": {
                        "foreground": "#05C3BA",
                    }
                },
                {
                    "scope": "string.regexp",
                    "settings": {
                        "foreground": "#D69D85",
                    }
                },
                {
                    "scope": "constant.character.escape.regexp",
                    "settings": {
                        "foreground": "#D69D85",
                    }
                },
                {
                    "scope": "constant.other.escape.regexp",
                    "settings": {
                        "foreground": "#FFD68F",
                    }
                },
            ]
        },
        "[Default Light+]": {
            "textMateRules": [
                {
                    "scope": "entity.name.namespace",
                    "settings": {
                        "foreground": "#222222"
                    }
                },
                {
                    "scope": "entity.name.variable.field",
                    "settings": {
                        "foreground": "#222222"
                    }
                },
                {
                    "scope": "variable.other.property",
                    "settings": {
                        "foreground": "#222222"
                    }
                },
                {
                    "scope": "variable.other.constant",
                    "settings": {
                        "foreground": "#222222",
                    }
                },
                {
                    "scope": "variable.other.enummember",
                    "settings": {
                        "foreground": "#222222"
                    }
                },
                {
                    "scope": "entity.name.type.interface",
                    "settings": {
                        "foreground": "#267f99"
                    }
                },
                {
                    "scope": "entity.name.type.enum",
                    "settings": {
                        "foreground": "#267f99"
                    }
                },
                {
                    "scope": "entity.name.type.parameter",
                    "settings": {
                        "foreground": "#267f99"
                    }
                },
                {
                    "scope": "entity.name.type.struct",
                    "settings": {
                        "foreground": "#267f99",
                    }
                },
                {
                    "scope": "entity.name.function.extension",
                    "settings": {
                        "foreground": "#795E26",
                    }
                },
                {
                    "scope": "comment.documentation",
                    "settings": {
                        "foreground": "#008000",
                    }
                },
                {
                    "scope": "comment.documentation.attribute",
                    "settings": {
                        "foreground": "#282828",
                    }
                },
                {
                    "scope": "comment.documentation.cdata",
                    "settings": {
                        "foreground": "#808080",
                    }
                },
                {
                    "scope": "comment.documentation.delimiter",
                    "settings": {
                        "foreground": "#808080",
                    }
                },
                {
                    "scope": "comment.documentation.name",
                    "settings": {
                        "foreground": "#808080",
                    }
                }
            ]
        }
    },

NTaylorMullen · 2020-03-16T02:16:11Z

@ryanbrandenburg since you own our semantic classification you should take a look at this as well. It will also enable us to call through to Roslyn for semantic colorization as well 😄

codecov · 2020-03-16T02:38:27Z

Codecov Report

Merging #3667 into master will increase coverage by 0.02%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #3667      +/-   ##
==========================================
+ Coverage   87.01%   87.03%   +0.02%     
==========================================
  Files          59       59              
  Lines        1779     1782       +3     
  Branches      207      207              
==========================================
+ Hits         1548     1551       +3     
  Misses        176      176              
  Partials       55       55

Flag	Coverage Δ
#integration	`100.00% <ø> (ø)`
#unit	`87.03% <100.00%> (+0.02%)`	⬆️

Impacted Files	Coverage Δ
src/omnisharp/options.ts	`94.31% <100.00%> (+0.13%)`	⬆️
src/omnisharp/protocol.ts	`77.34% <100.00%> (+0.17%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 65168f8...a2fb93e. Read the comment docs.

src/omnisharp/extension.ts

NTaylorMullen · 2020-03-16T02:41:50Z

src/features/semanticTokensProvider.ts

+        const semanticSpans = response.Spans;
+
+        const builder = new vscode.SemanticTokensBuilder();
+        for (let span of semanticSpans) {


Maybe I'm misunderstanding the code but why not have the server just return the LSP compatible type vs. re-building it here?

I was just following the typescript extension as an example where they request information from their server then perform some post processing. I'll have to think more on whether to keep it like this.

Ah, TypeScript hasn't moved fully over to LSP yet which is why they follow that model. They plan to move fully over to LSP

src/features/semanticTokensProvider.ts

NTaylorMullen · 2020-03-16T02:44:01Z

src/features/semanticTokensProvider.ts

+export default class SemanticTokensProvider extends AbstractProvider implements vscode.DocumentSemanticTokensProvider, vscode.DocumentRangeSemanticTokensProvider {
+
+    getLegend(): vscode.SemanticTokensLegend {
+        return new vscode.SemanticTokensLegend(tokenTypes, tokenModifiers);


The server's typically the one that defines the legend. Why re-create it on the client (vs. querying it)?

Yeah goes back to whether the OmniSharp endpoint should just implement the LSP return types. I'll think on it.

JoeRobich · 2020-05-10T05:29:53Z

@NTaylorMullen Would like to include this in the next release. We'll consider it experimental (defaulted off) and collect feedback about the performance. Ultimately we may implement an API that is closer to the LSP, but that may wait until Roslyn implements a classification service that returns tokens and modifiers.

filipw

NTaylorMullen · 2020-05-11T17:06:32Z

Ultimately we may implement an API that is closer to the LSP, but that may wait until Roslyn implements a classification service that returns tokens and modifiers.

Ah, when that comes to fruition that'll enable us to add semantic C# colorization to Razor's C# pieces. Is that something that's tracked somewhere? FYI @ryanbrandenburg

We'll consider it experimental (defaulted off) and collect feedback about the performance.

What type of switch do you plan to put it behind? Asking because our TagHelper semantic highlighting is cheap and is something we'd have "on" all the time however if we were to semantically highlight our C# pieces of the document we'd want it behind a similar switch to ensure it's only active when yours is active.

JoeRobich · 2020-05-11T20:55:05Z

Is that something that's tracked somewhere?

We didn't have an exact issue tracking this, so I made a proposal. dotnet/roslyn#44155

What type of switch do you plan to put it behind?

There is a csharp.semanticHighlighting.enabled setting which is available off of the Options class.

JoeRobich · 2020-05-11T20:56:24Z

Although GitHub checks didn't update, this is the passing Travis CI for this PR - https://travis-ci.org/github/OmniSharp/omnisharp-vscode/builds/685827799

Merging

JoeRobich · 2020-05-11T21:08:36Z

for good measure this is the passing master build following this merge https://travis-ci.org/github/OmniSharp/omnisharp-vscode/builds/685844034

JoeRobich requested review from NTaylorMullen and filipw March 15, 2020 00:50

NTaylorMullen reviewed Mar 16, 2020

View reviewed changes

JoeRobich mentioned this pull request Mar 16, 2020

Semantic Tokens API microsoft/vscode#86415

Closed

JoeRobich added the Blocked label Mar 16, 2020

JoeRobich mentioned this pull request Mar 18, 2020

Theme Changes to better align with Visual Studio Light and Dark themes microsoft/vscode#92965

Closed

JoeRobich force-pushed the semantic-highlight branch 2 times, most recently from 1e2c4d7 to 83ecd67 Compare May 10, 2020 05:16

JoeRobich removed the Blocked label May 10, 2020

filipw approved these changes May 10, 2020

View reviewed changes

filipw mentioned this pull request May 10, 2020

Nullable and IAsyncEnumerable break syntax highlighting OmniSharp/omnisharp-roslyn#1786

Closed

JoeRobich added 8 commits May 11, 2020 12:51

Add experimental Semantic Highlighter

7096c6c

Update the endpoint

2b59ddc

Use modifiers returned from end point

201366a

Revert vscode dependency version bump

9945090

Do not register during integration tests

66d72b1

Ensure document version hasn't changed

8c2f340

Update to the released semantic token apis.

5295a55

Add XMLDocComment classifications

29b3f6c

JoeRobich force-pushed the semantic-highlight branch 2 times, most recently from 89beb40 to a00eec9 Compare May 11, 2020 20:03

JoeRobich added 2 commits May 11, 2020 13:13

Add configuration setting for enabling/disabling semantic highlight

931cf93

Add integration test for SemanticTokensProvider

a2fb93e

JoeRobich force-pushed the semantic-highlight branch from a00eec9 to a2fb93e Compare May 11, 2020 20:13

JoeRobich merged commit 4b249d1 into dotnet:master May 11, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add experimental Semantic Highlighter #3667

Add experimental Semantic Highlighter #3667

JoeRobich commented Mar 15, 2020 •

edited

Loading

NTaylorMullen commented Mar 16, 2020

codecov bot commented Mar 16, 2020 •

edited

Loading

NTaylorMullen Mar 16, 2020

JoeRobich Mar 16, 2020

NTaylorMullen Mar 16, 2020

NTaylorMullen Mar 16, 2020

JoeRobich Mar 16, 2020

JoeRobich commented May 10, 2020

filipw left a comment

NTaylorMullen commented May 11, 2020

JoeRobich commented May 11, 2020

JoeRobich commented May 11, 2020

JoeRobich commented May 11, 2020

Add experimental Semantic Highlighter #3667

Add experimental Semantic Highlighter #3667

Conversation

JoeRobich commented Mar 15, 2020 • edited Loading

NTaylorMullen commented Mar 16, 2020

codecov bot commented Mar 16, 2020 • edited Loading

Codecov Report

NTaylorMullen Mar 16, 2020

Choose a reason for hiding this comment

JoeRobich Mar 16, 2020

Choose a reason for hiding this comment

NTaylorMullen Mar 16, 2020

Choose a reason for hiding this comment

NTaylorMullen Mar 16, 2020

Choose a reason for hiding this comment

JoeRobich Mar 16, 2020

Choose a reason for hiding this comment

JoeRobich commented May 10, 2020

filipw left a comment

Choose a reason for hiding this comment

NTaylorMullen commented May 11, 2020

JoeRobich commented May 11, 2020

JoeRobich commented May 11, 2020

JoeRobich commented May 11, 2020

JoeRobich commented Mar 15, 2020 •

edited

Loading

codecov bot commented Mar 16, 2020 •

edited

Loading