C++20 keywords don't seem to be highlighted #396

cjdb · 2019-10-11T01:52:06Z

Checklist

This problem exists even with the setting "C_Cpp.enhancedColorization": "Disabled"
This bug exists for C
This bug exists for C++
This bug exists for Objective-C
This bug exists for Objective-C++

The code with a problem is:

module example;
import <vector>
import some.other.module;

export template<typename From, typename To>
concept convertible_to =
	std::is_convertible_v<From, To> and
	requires(From (&f)()) {
		static_cast<To>(f());
	};

In the above example, module, import, <vector>, some.other.module, concept, and requires should all be highlighted, but they're not.

It looks like:

Theme: peaceshi

Theme: XD

Theme: One Monokai

It should look like:

module, import, concept, and requires should all be highlighted like the other keywords. (Note: module is contextually-dependent.)
<vector> and some.other.module should probably look like <vector> in #include <vector>

The text was updated successfully, but these errors were encountered:

cjdb · 2019-10-11T02:01:12Z

I'd be okay with some.other.module not being highlighted.

matter123 · 2019-10-11T02:07:16Z

Duplicate of #302. Current focus is on published standards.

matter123 · 2019-10-11T02:15:44Z

Just keyword highlighting, no context awareness, could be added in easily enough. Import as an alternative to #include should also be easy enough to add.

cjdb · 2019-10-11T02:23:42Z

I've been trying to work out how to do this locally, but without success. Looks like the above words are already in the JSON file, which has confused me.

matter123 · 2019-10-11T02:52:36Z

Often those are in negative lookbehinds and lookaheads.

import: the import keyword needs to be pulled out of the check for #, But if the semicolon is required it might be best if its pulled into its own pattern.

Adding keywords: This should be just as simple as adding the correct flags to the keywords in https://github.com/jeff-hykin/cpp-textmate-grammar/blob/master/source/languages/cpp/tokens.rb#L261-L274

cjdb · 2019-10-11T03:04:42Z

Thanks. I was definitely barking up the wrong tree, trying to add what's below to generate.rb.

grammar[:concept_definition] = Pattern.new(
        tag_as: "meta.declaration.concept",
        should_fully_match: ["concept A = B;"]
        match: Pattern.new(
                match:/concept/,
                tag_as: "keyword.other.concept",
            ).maybe(@spaces).then(
                identifier
            ).maybe(@spaces).then(
                assignment_operator
            ).maybe(@spaces).then(
                /[^;]+/
            ).maybe(@spaces).then(@semicolon.or(/\n/)),
    )

matter123 · 2019-10-11T03:09:48Z

Looks good, couple changes.

use .then(std_space) rather than .maybe(@spaces). This is for a couple reasons:
- std_space supports inline comments
- std_space is automatically optional
- .maybe(@spaces) can catastrophically backtrack.
.then(/[^;]+/) should be .oneOrMoreOf(match: /[^;]/, dont_backtrack?: true)
use @end_of_line instead of /\n/
You still want a fallback to highlight /concept/ if the context aware pattern fails.

import support is now correct

cjdb · 2019-10-11T03:40:07Z

Do concept, requires, and import need logic like what's above? I get module probably does, but the others aren't context-dependent keywords.

matter123 · 2019-10-11T03:48:39Z

If a keyword doesn't influence any other highlighting, then all that is need is to add the correct flags inside tokens.rb. In the new PR, I added support for import.

cjdb · 2019-10-11T04:11:49Z

I think this is a rookie mistake, but I'm having issues with npm run build.

cjdb: npm test

> better-syntax@1.0.0 pretest /mnt/c/Users/Chris/projects/cpp-textmate-grammar
> npm run build


> better-syntax@1.0.0 build /mnt/c/Users/Chris/projects/cpp-textmate-grammar
> ruby scripts/generate.rb

scripts/generate.rb:6: warning: Insecure world writable dir /mnt/c/Users/Chris/projects/cpp-textmate-grammar/node_modules/.bin in PATH, mode 040777
Traceback (most recent call last):
        4: from /mnt/c/Users/Chris/projects/cpp-textmate-grammar/source/languages/c/generate.rb:3:in `<main>'
        3: from /mnt/c/Users/Chris/projects/cpp-textmate-grammar/source/languages/c/generate.rb:3:in `require_relative'
        2: from /mnt/c/Users/Chris/projects/cpp-textmate-grammar/source/textmate_tools.rb:4:in `<top (required)>'
        1: from /usr/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
/usr/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require': cannot load such file -- deep_clone (LoadError)


Generating the syntax for 'c' failed: 256
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! better-syntax@1.0.0 build: `ruby scripts/generate.rb`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the better-syntax@1.0.0 build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/cjdb/.npm/_logs/2019-10-11T04_10_51_315Z-debug.log
npm ERR! Test failed.  See above for more details.

Are there any prerequisites I should be installing? I've run npm install and npm install top to no avail.

matter123 · 2019-10-11T04:13:23Z

Yes bundle install, I should probably add a post install hook.

Partial fix for #396

jeff-hykin · 2019-10-15T23:02:13Z

Yes bundle install, I should probably add a post install hook.

I went ahead and added that to the CONTRIBUTING.md for next time. Sorry, about that @cjdb, and thanks for taking the initiative to get things running on your own machine.

cjdb · 2019-10-18T00:33:20Z

I've noticed that since pulling 9f50b88, my custom requires highlighting has stopped working. Highlighting doesn't show even when it's configured identical to consteval, but interestingly, it doesn't highlight at all when I try to use it like a normal function.

matter123 · 2019-10-18T01:18:37Z

It is expected that requires is not usable as a function name due to, avoid_invalid_function_names = @cpp_tokens.lookBehindToAvoidWordsThat(:isWord, not(:isPreprocessorDirective), not(:isValidFunctionName)).

Can you share what you have for the requires highlighting? As well as what context you added it to.

cjdb · 2019-10-18T01:25:42Z

Can you share what you have for the requires highlighting?

# source/language/cpp/tokens.rb
{ representation: "consteval", name: "consteval", isLambdaSpecifier: true },
{ representation: "requires",  name: "requires",  isLambdaSpecifier: true },

Same as consteval for now. Figured I'd start small, and work my way up.

As well as what context you added it to.

[]() consteval x {}; // consteval highlighted
[]() requires x {}; // requires not highlighted

matter123 · 2019-10-18T01:32:16Z

If you are using the debugger could you check that you are using the "Launch and Build All" configuration, the default "Launch Extension" does not usually rebuild the syntax files.

cjdb · 2019-10-18T01:34:56Z

I've been manually running npm test, but I just tried Launch and Build All: no difference.

matter123 · 2019-10-18T01:51:10Z

That is very strange, If I add the requires token directly after consteval (line 221) everything seems to work.

Could you remove the other requires definition (on line 263) just to be sure?

Is the lambda definition inside of a macro?

@jeff-hykin Any thoughts on why it wouldn't work?

cjdb · 2019-10-18T01:53:41Z

Still nothing. Is there any chance that the customised file isn't being loaded, and the default one is?

matter123 · 2019-10-18T02:00:06Z

There shouldn't be, npm test has no knowledge of what vscode loads normally, and the extension development host replaces the extension, so that the default one should never be loaded.

matter123 · 2019-10-18T02:04:20Z

Is the pattern being recognized as a lambda? (Is the scope at { meta.function.definition.body.lambda.cpp when using Developer: Inspect TM Scopes

matter123 · 2019-10-18T02:26:22Z

Could you try to reproduce this from a clean clone of the repo to try to determine if this is a build issue or if other changes might be affecting it?

cjdb · 2019-10-18T03:44:03Z

Could you try to reproduce this from a clean clone of the repo to try to determine if this is a build issue or if other changes might be affecting it?

This didn't fix it. Perhaps we should jump on an IM channel? I'm available on cpplang.slack or #include<C++> Discord under @cjdb.

Is the pattern being recognized as a lambda? (Is the scope at { meta.function.definition.body.lambda.cpp when using Developer: Inspect TM Scopes

See attached.

jeff-hykin · 2019-10-18T14:05:08Z

I've been manually running npm test, but I just tried Launch and Build All: no difference.

Is it opening a separate window?
Running npm test will generate the files, but it won't change any visuals. To see the changes VS Code has to compile the extension, create a new window, and then sudo-install the newly compiled extension inside that window. And most times: when a change is made to the code the entire new-window needs to be reloaded.

Then new visuals and Developer: Inspect TM Scopes will only work inside that new window. Everywhere else will be the old version.

This didn't fix it. Perhaps we should jump on an IM channel?

Not a bad idea. Although if other people have this problem, we do want it to be findable.

jeff-hykin · 2020-01-21T19:07:45Z

just an update on this problem: in v1.14.20 (pushed just now) should add at least basic highlighting for those keywords

HighCommander4 · 2020-01-21T22:34:06Z

just an update on this problem: in v1.14.20 (pushed just now) should add at least basic highlighting for those keywords

Awesome, thanks!

(Note: it does not highlight the second requires keyword from this example (the one that comes from a requires-clause rather than a requires-expression), but perhaps that's a known issue.)

jeff-hykin · 2020-01-22T04:23:34Z

yeah... thanks for pointing that out. That case in particular is more of a templating problem see
#246: Template definition needs multiple fixes

soroshsabz · 2021-10-15T00:45:50Z

related to #573
related to #575

RedHolms · 2024-09-21T19:27:34Z

Still open, and still doesn't work

matter123 closed this as completed Oct 11, 2019

matter123 added duplicate This issue or pull request already exists Syntax: C++ issue exists for C++ ✨ Enhancement New feature or request labels Oct 11, 2019

matter123 reopened this Oct 11, 2019

matter123 added a commit that referenced this issue Oct 11, 2019

Fix for some items in #396

6c1abe4

import support is now correct

matter123 added a commit that referenced this issue Oct 13, 2019

Add import statments(#397)

9f50b88

Partial fix for #396

HighCommander4 mentioned this issue Jan 21, 2020

Highlight new C++20 Concepts keywords ("concept" and "requires") #441

Closed

Colengms mentioned this issue May 21, 2020

concept/requires keywords are not highlighted microsoft/vscode-cpptools#5546

Closed

This was referenced Oct 15, 2021

Add consteval keyword highlighted. (fixes #573) #574

Merged

Add constinit keyword highlighted. (fixes #575) #576

Merged

jgcodes2020 mentioned this issue Jan 1, 2022

C++20: consteval and constinit keywords #583

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

C++20 keywords don't seem to be highlighted #396

C++20 keywords don't seem to be highlighted #396

cjdb commented Oct 11, 2019 •

edited

Loading

cjdb commented Oct 11, 2019

matter123 commented Oct 11, 2019

matter123 commented Oct 11, 2019

cjdb commented Oct 11, 2019

matter123 commented Oct 11, 2019 •

edited

Loading

cjdb commented Oct 11, 2019

matter123 commented Oct 11, 2019 •

edited

Loading

cjdb commented Oct 11, 2019

matter123 commented Oct 11, 2019 •

edited

Loading

cjdb commented Oct 11, 2019

matter123 commented Oct 11, 2019

jeff-hykin commented Oct 15, 2019

cjdb commented Oct 18, 2019

matter123 commented Oct 18, 2019

cjdb commented Oct 18, 2019

matter123 commented Oct 18, 2019

cjdb commented Oct 18, 2019

matter123 commented Oct 18, 2019 •

edited

Loading

cjdb commented Oct 18, 2019

matter123 commented Oct 18, 2019

matter123 commented Oct 18, 2019

matter123 commented Oct 18, 2019

cjdb commented Oct 18, 2019

jeff-hykin commented Oct 18, 2019

jeff-hykin commented Jan 21, 2020

HighCommander4 commented Jan 21, 2020

jeff-hykin commented Jan 22, 2020 •

edited

Loading

soroshsabz commented Oct 15, 2021

RedHolms commented Sep 21, 2024

C++20 keywords don't seem to be highlighted #396

C++20 keywords don't seem to be highlighted #396

Comments

cjdb commented Oct 11, 2019 • edited Loading

The code with a problem is:

It looks like:

It should look like:

cjdb commented Oct 11, 2019

matter123 commented Oct 11, 2019

matter123 commented Oct 11, 2019

cjdb commented Oct 11, 2019

matter123 commented Oct 11, 2019 • edited Loading

cjdb commented Oct 11, 2019

matter123 commented Oct 11, 2019 • edited Loading

cjdb commented Oct 11, 2019

matter123 commented Oct 11, 2019 • edited Loading

cjdb commented Oct 11, 2019

matter123 commented Oct 11, 2019

jeff-hykin commented Oct 15, 2019

cjdb commented Oct 18, 2019

matter123 commented Oct 18, 2019

cjdb commented Oct 18, 2019

matter123 commented Oct 18, 2019

cjdb commented Oct 18, 2019

matter123 commented Oct 18, 2019 • edited Loading

cjdb commented Oct 18, 2019

matter123 commented Oct 18, 2019

matter123 commented Oct 18, 2019

matter123 commented Oct 18, 2019

cjdb commented Oct 18, 2019

jeff-hykin commented Oct 18, 2019

jeff-hykin commented Jan 21, 2020

HighCommander4 commented Jan 21, 2020

jeff-hykin commented Jan 22, 2020 • edited Loading

soroshsabz commented Oct 15, 2021

RedHolms commented Sep 21, 2024

cjdb commented Oct 11, 2019 •

edited

Loading

matter123 commented Oct 11, 2019 •

edited

Loading

matter123 commented Oct 11, 2019 •

edited

Loading

matter123 commented Oct 11, 2019 •

edited

Loading

matter123 commented Oct 18, 2019 •

edited

Loading

jeff-hykin commented Jan 22, 2020 •

edited

Loading