Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Major Changes for Downstream #64

Open
alexr00 opened this issue Apr 9, 2019 · 12 comments
Open

Major Changes for Downstream #64

alexr00 opened this issue Apr 9, 2019 · 12 comments
Assignees
Labels
📖 Clarification Discussions on behavior of the tagging workflow Part of an overview of how/what things are being accomplished

Comments

@alexr00
Copy link

alexr00 commented Apr 9, 2019

It looks like new used to be keyword.control.cpp. Now it is keyword.operator.memory.new.cpp. Is this intentional? If it's intentional, we can add the it into the themes in VS Code, but ideally there wouldn't be too many changes like this since it requires constant updating of the themes.

@alexr00
Copy link
Author

alexr00 commented Apr 9, 2019

Original issue: microsoft/vscode#71821

@jeff-hykin
Copy link
Owner

jeff-hykin commented Apr 9, 2019

Yes it was intentional, and I agree we will try to keep changes like this to a minimum since many themes already setup with the old way.

From here on out, I'll put any similar changes on this post.

C++ Log as of August 12th 2019

  • Helper-tags have been added to make tag changes much easier to maintain. None of these change/replace existing tags
    • (1.7.0) keyword.operator.functionlike.cpp keyword.operator
      will match all of the operators like alignas, sizeof, etc that require parentheses
    • (1.7.0) keyword.operator.wordlike.cpp keyword.operator
      will match all of the operators that use alpha numeric characters (that are not functionlike)
    • (1.7.0) memory keyword.operator
      will match both keyword.operator.new and keyword.operator.delete
  • new and delete changed
    • (1.0.0) keyword.control
    • (1.4.0) keyword.operator.memory.new
    • (1.7.0) keyword.operator.new and keyword.operator.delete
  • The using keyword changed
    • (1.0.0) keyword.control
    • (1.4.0) keyword.other.using
  • and, or, xor, etc have received additional specificity
    • (1.7.0) match them with alias keyword.operator
    • (1.11.7) match them with keyword.operator.wordlike keyword.operator or keyword.operator.or
  • The operator keyword was changed
    • (1.0.0) keyword.control
    • (1.5.0) keyword.other.operator
  • Member access received additional scopes
    • (1.4.8) variable.other.object.accessor
      • To effectively revert this set the color of this tag to be the same as source.cpp
  • Member variables have changed
    • (1.0.0) variable.other.member
    • (1.4.8) variable.other.object.property (middle) and variable.other.property (end)
  • The namespace {}'s have changed
    • (1.0.0) punctuation.definition.scope
    • (1.7.0) punctuation.section.block.begin.bracket.curly.namespace.cpp punctuation.section.block.end.bracket.curly.namespace.cpp
      • This change is because class/struct/enum/union/extern all use that for their brackets. Namespace brackets are the only anti-pattern
  • decltype is now tagged
    • (1.6.0) both storage.type.decltype and keyword.other.decltype
  • The class, struct, union, enum, extern, namespace pattern(s) were overhauled and standardized
    • (1.0.0) meta.class-struct-block, meta.extern-block, meta.namespace-block
    • (1.7.0) changed to
      • meta.block.class
      • meta.block.struct
      • meta.block.union
      • meta.block.enum
      • meta.block.extern
      • meta.block.namespace
  • scope of namespaces has changed
    • (1.8.2) entity.name.type.namespace
    • (1.9.1) entity.name.namespace
  • pthread types have changed
    • (1.0) support.type.pthread
    • (1.8.5) support.type.posix-reserved.pthread
  • change to ()'s
    • (1.0) punctuation.section.parens
    • (1.8.6) meta.parens.c
    • [ subject to change in future, it will stay meta, but might be meta.block or similar ]
  • function pointers
    • (1.0) no tagging
    • (1.8.14) experimental tagging inside of structs/classes only
    • (1.9.0) tagging inside of function parameters
  • pointer and reference
    • (1.0) keyword.operator
    • (1.8.14) storage.modifier.pointer and storage.modifier.reference
  • scope resolution identifiers
    • (1.0?) entity.name.type.namespace.scope-resolution
    • (1.9.0)
      • entity.name.scope-resolution
      • entity.name.scope-resolution.function.call
      • entity.name.scope-resolution.function.definition
      • entity.name.scope-resolution.function.definition.operator-overload
      • entity.name.scope-resolution.namespace.alias
      • entity.name.scope-resolution.namespace.block
      • entity.name.scope-resolution.namespace.using
      • entity.name.scope-resolution.template.call
      • entity.name.scope-resolution.template.definition
  • scope resolution punctuation ::
    • (1.0) punctuation.separator.namespace.access
    • (1.9.0)
      • punctuation.separator.scope-resolution
      • punctuation.separator.scope-resolution.function.call
      • punctuation.separator.scope-resolution.function.definition
      • punctuation.separator.scope-resolution.function.definition.operator-overload
      • punctuation.separator.scope-resolution.namespace.alias
      • punctuation.separator.scope-resolution.namespace.block
      • punctuation.separator.scope-resolution.namespace.using
      • punctuation.separator.scope-resolution.template.call
      • punctuation.separator.scope-resolution.template.definition
    • (1.9.2) planned to add punctuation.separator.namespace.access for better backwards compatibility
  • template call angle brackets
    • (1.0) keyword.operator.comparision
    • (1.9.0) more of them are now tagged as
      • punctuation.section.angle-brackets.begin.template.call
      • punctuation.section.angle-brackets.end.template.call
      • punctuation.section.angle-brackets.begin.template.definition
      • punctuation.section.angle-brackets.end.template.definition
  • macro parentheses
    • (1.0) punctuation.section.arguments.begin.bracket.round.cpp
    • (1.0) punctuation.section.arguments.end.bracket.round.cpp
    • (1.13.0) punctuation.definition.parameters.begin.preprocessor
    • (1.13.0) punctuation.definition.parameters.end.preprocessor

@jeff-hykin jeff-hykin self-assigned this Apr 9, 2019
@jeff-hykin jeff-hykin added the 📖 Clarification Discussions on behavior of the tagging label Apr 9, 2019
@jeff-hykin
Copy link
Owner

jeff-hykin commented Apr 9, 2019

The reason it was changed is because new and delete are completely unrelated to control flow. I picked operator since that's what what I believed it was classified under the C++ spec.

@matter123 pointed out in his pull request its only an operator under a special case. Additionally its debatable that the coloration would be improved if it were labeled with keyword.other.

@alexr00 should we go ahead and make the change from keyword.operator to keyword.other?

@jeff-hykin jeff-hykin changed the title new changed from keyword.control.cpp to keyword.operator.memory.new.cpp Log of Major Changes to Existing Patterns Apr 9, 2019
@alexr00
Copy link
Author

alexr00 commented Apr 9, 2019

Having the big changes here is perfect, thank you. I took a quick look at how other languages handle new:
coffee
keyword.operator.new.coffee

JS
keyword.operator.new.js

groovy
keyword.control.new.groovy

java
keyword.control.new.java

There isn't a real consensus. Based on what the VS Code themes already have, keyword.operator.new would be good. keyword.other, like keyword.operator.memory would still require changes to the theme. I'm happy to make minor updates to themes though. If you prefer keyword.operator.memory more, then feel free to leave it and I'll make the update to include it here and in the equivalent places in the other themes.

@matter123
Copy link
Collaborator

matter123 commented Apr 9, 2019

keyword.operator.new is probably the best. I noticed on [expr.new]/4 the standard refers to the new operator. Though change is still needed for keyword.operator.delete

@jeff-hykin
Copy link
Owner

Thats interesting @alexr00 I'll try to get a complete list of changes by the end of tonight. There is a file in this repo syntax_examples.cpp it is designed to visually debug the syntax highlighter changes. You can probably use it to quickly see how everything looks.

@jeff-hykin
Copy link
Owner

jeff-hykin commented Apr 10, 2019

@alexr00 The list isn't 100% complete, but I believe it covers everything that would be considered an issue. Many of the changes were made tonight to try to get all of the changes done now. I probably won't be able to make changes for about a week or more, but I'll be around daily for GitHub communication.

Do note that these changes are C++ exclusive. We are working on an ideal way to port them to C.

I believe the new keyword.operator.wordlike, keyword.operator.functionlike should take care of most of the issues.

@jeff-hykin jeff-hykin added the workflow Part of an overview of how/what things are being accomplished label Apr 11, 2019
@matter123 matter123 pinned this issue Apr 11, 2019
@jeff-hykin
Copy link
Owner

Today I added a tags.txt, that gets auto generated with the syntax. It just sorts and lists out all of the tags that are in the grammar, it should be useful for keeping track of when things are added and removed. @matter123

@jeff-hykin
Copy link
Owner

jeff-hykin commented May 29, 2019

@alexr00 The Objective-C and Objective-C++ are fully restored/fixed.
Yesterday we expanded the repo's structure to accommodate multiple languages. We added both the Objective-C and Objective-C++ syntaxes along with integration tests for both of them. Neither of them are dependent on C and C++ repos anymore. There's also been some work done on improving the Objective-C++ syntax, but to be on the safe side those changes are not included.

Other Info About the Problem

If you want to know more about why this happened in the first place, see here: #107 (comment)

Once the updated Objective-C and Objective-C++ are in VS Code, we can actually instantly fix some long standing bugs related to atom/language-c#146

Scope Change Guide

To make pulling the latest changes easy, here's a scope-change guide for the "dark+" and "dark" themes.

Making a pattern specific to C++

I'd recommend adding
source.cpp keyword.operator.new
rather than
keyword.operator.new.cpp
The second way makes it where additional specificity like keyword.operator.new.overload.cpp will break the theme

Might want to change "Control flow keywords"

The only scope in the "Control flow keywords" that is control flow is keyword.control, all of the others (keyword.operator.new.cpp, keyword.operator.delete, keyword.other.using, keyword.other.operator) are unrelated to control flow. So you might want to rename that group to something like "Special Keywords"

Scope Names

dark plus
    // probably make these white #d4d4d4 (same as keyword.operator)
        "storage.modifier.pointer",
        "storage.modifier.reference"
    // probably put these with the "Control flow keywords"/"Special Keywords"
        "keyword.operator.functionlike keyword.operator", // typeid(), alignas(), etc
        "keyword.operator.wordlike keyword.operator",  // and, or, xor, not, etc
    // C++ attributes, maybe color them purple, blue, or light blue
        "entity.other.attribute",
        "punctuation.section.attribute.begin",
        "punctuation.section.attribute.end",
    // might want to put the overloadee in "Function declarations"
        "entity.name.operator.overloadee"
    // might want to add this to "Types declaration and references"
    // since its almost always a class or namespace that is the scope-resolution
        "entity.name.scope-resolution"

dark theme:
    // probably want to color blue:
        "support.type"
        "keyword.operator.wordlike keyword.operator"
    // probably make these white #d4d4d4 (same as keyword.operator)
        "storage.modifier.pointer"
        "storage.modifier.reference"

@alexr00
Copy link
Author

alexr00 commented May 31, 2019

@jeff-hykin thank you for the detailed description of the issue and for all the fixes and suggestions! We are planning on releasing soon, and I want all these changes to be in insiders for a while before they go out in stable. As soon as we have branched for release (probably in the next couple week days) I'll bring in your Objective C and Objective C++ changes.

@jeff-hykin
Copy link
Owner

Great 👍 and I agree, I'd be more comfortable with them staying in the insiders branch for awhile.

@jeff-hykin jeff-hykin changed the title Log of Major Changes to Existing Patterns Major Changes for Downstream Apr 16, 2024
@jeff-hykin
Copy link
Owner

Starting at v1.22.0 you might get downstream complaints about no SQL highlighting.

  • v1.22.0 removes the source.sql since it creates frequent unfixable cascading breakages. The side effect is that no SQL keywords will be highlighted inside of raw strings.
  • Highlighting SQL is still possible, as raw SQL strings are given the string.quoted.double.raw.sql and/or string.quoted.double.raw.SQL textmate scope. Extensions/editors can target that scope to provide SQL highlighting for those that still want it. Not great I know, but I think its better than defaulting to cascading breakages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
📖 Clarification Discussions on behavior of the tagging workflow Part of an overview of how/what things are being accomplished
Projects
None yet
Development

No branches or pull requests

3 participants