-
Notifications
You must be signed in to change notification settings - Fork 401
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix multiline semantic highlighting for 'record' #2789
Conversation
d2931da
to
e1d57ae
Compare
I believe this fixes redhat-developer/vscode-java#1442 as well, though I couldn't reproduce the automatic line-splitting for inner records mentioned, even with version 0.61.0. The highlight colours are now correct. |
e1d57ae
to
cb2222c
Compare
It looks very reasonable. I'll just have a look at some other way to properly apply the same colouring as |
When calling Currently, you first add the In fact, you can see that the tokens have been incorrectly encoded, because there is a test failure for Here is the correct visitation order: https://github.com/eclipse-jdt/eclipse.jdt.core/blob/4c26c0761103220355168e77cd6e556274b131a7/org.eclipse.jdt.core/dom/org/eclipse/jdt/core/dom/RecordDeclaration.java#L305-L312 The @Override
public boolean visit(RecordDeclaration node) {
acceptNode(node.getJavadoc());
acceptNodeList(node.modifiers());
// Here goes the code that calls addToken for the "record" keyword
acceptNode(node.getName());
acceptNodeList(node.typeParameters());
acceptNodeList(node.recordComponents());
acceptNodeList(node.superInterfaceTypes());
acceptNodeList(node.bodyDeclarations());
return false;
} |
Also, what's the reason for adding semantic token modifiers such as Regarding this code comment: The |
The But that feels like a separate discussion. The fact is that currently, the |
Signed-off-by: Hope Hadfield <hhadfiel@redhat.com>
cb2222c
to
a219b76
Compare
@0dinD Thanks for the input. I've fixed the order of the token addition, removed the modifiers for the |
I suspect the choice for Thanks for the reference to @0dinD I just have one question about the approach. If the semantic tokens must be added in order of appearance in the document, why don't we just sort them at the very end, instead of making the reading process more convoluted ? I can't imagine we're saving that much time by avoiding the additional sort at the end. |
The TextMate grammar we use was originally created for the Atom editor, but yeah, perhaps the authors took some inspiration from Eclipse. Whatever the reason might be, I think it's still the wrong solution. If the goal was to apply the same color to the modifiers and the
That thought has occurred to me. Sorting the list of tokens would allow us to visit nodes without needing to worry about ordering, but sorting a list with thousands of items (possibly at a high rate when the user is typing, which causes many token requests) will have a performance cost. On the one handOriginally (before I contributed patches for the semantic tokens), ordering was not a problem we had to think about, because the Basically, the AST is already sorted (by nature). It feels wrong to traverse the tree in the wrong order, and then "cover it up" by sorting the result. Like, why not just do an in-order traversal? Also, due to the nature of sorting, the performance impact would not scale linearly. For extremely large source files with a large number of semantic tokens, the performance impact could grow quite large. Of course, it's hard to say too much about this performance impact without actually measuring it. Nowadays, we do have to worry about visitation order, because we sometimes choose not to delegate to
I don't know exactly what you're referring to here, but keep in mind that even if we sort the list, we still have to call In this specific case with the On the other handI do agree that sorting the list would make the code a bit easier to maintain. If it turns out that the performance cost is acceptable, it's probably a good idea. I suspect (without having done much testing) that, for normal-sized source files, the cost of sorting the list is dwarfed by the costs involved with waiting for the shared AST provider and other document lifecycle jobs. On the other hand, I have some ideas for how those performance costs could be removed, which I would like to explore in the future. In order to get a relevant performance metric, it would probably be good to look at the performance of |
Great, this PR looks good to me now! On paper, I haven't had time to test it. It looks like the CI tests have not run for this PR (or at least, I can't see them linked here on GitHub, all I see is CodeQL and ECA). Do you have any idea why that might be @rgrunber? Maybe because this PR was a draft PR at first, and then got converted into a "normal" PR? |
Test this please. We were running into https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/3449#note_1196262 due to the org migration. Should be fixed now. |
Ah, that explains it! Anyways, I tried the latest version of this PR out, and it works as expected. Looks good to me! |
I have some things to add to this discussion: #2789 (comment) After thinking about it some more, I realize that we could probably just change the implementation of Also, think we can probably get rid of most (if not all) of the manual child-visitation logic (the All of which is to say that I think there are a bunch of improvements that could be made to But that discussion is quite off-topic from this PR. I still think that the current implementation of this PR is good, given the current state of |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks correct to me. After I took some time to realize you're just copying the visitation order defined by the node for its children, it makes sense.
For now, I think we should be fine to respect visitation order. It seems only tokens not part of the AST model are at risk of being visited out of order (we may have access to their info at the "root" prior to visiting any children that come before it.
No, the risk of out-of-order visitation is introduced in all cases (even for tokens derived from AST nodes) where we avoid delegating the child visitation logic (returning As a reminder, here's one of my previous comments:
Here's an example where there is a risk of out-of-order visitation even though we are only visiting AST nodes: Lines 316 to 323 in 6fe5425
Now, I'm not sure if I explained what I meant very well in the above paragraph that I quoted. And I'm not sure if I agree with what I said anymore either way, now that I became aware of |
Fixes redhat-developer/vscode-java#1444
Before:
After: