[jade] Incorrect highlighting in strings with inline tags

[aeschli]

From @mrmlnc on January 4, 2016 12:54

VS Code version:
0.10.6

Code:

p #[code samp] — Regular text. #[samp This is sample text] more text.

Description:
If you try to write a dot in a inline tag (tag Interpolation), text highlighting changes.

Gif:

Copied from original issue: microsoft/vscode#1765

[aeschli]

This issue was moved to davidrios/jade-tmbundle#67

[aeschli]

From @mrmlnc on September 2, 2016 18:1

@aeschli, as I understand, this issue is related to the bug (feature) in vscode-textmate?

I ask this because I can fix Jade tmLanguage in VS Code (including other issues aka small refactoring). Does it make sense to do it?

UPD: And if it makes sense, then use PR or create demo extension?

[aeschli]

From @mrmlnc on September 5, 2016 22:49

@aeschli, you can see demo extension in VS Code 1.5.0

https://github.com/mrmlnc/vscode-jade-syntax

Correct highlighting for string with inline tags (fix for this issue) and correct highlighting of text in inline tags

Support for HTML comments

Both types (buf/unbuf) comments is a just comments

[aeschli]

We should fix that in https://github.com/Microsoft/vscode-textmate

[alexdima]

@mrmlnc @aeschli This issue has been moved so many times with automated tools and it contains screenshots of various inputs. I am not a Jade expert, so I do not understand what it is about.

Can you please help me out with a pointer to the grammar file, input text, what should happen vs what happens?

[mrmlnc]

@alexandrudima, yes, of course.

Clarification: that issue also reproduced in Atom with this grammar, but not Sublime Text 2 & 3.

Minimal code example

.class One #[span text.] Two text.

Current result

Tokens

• .class - token constant language js
• One - token
• # - token jade inline entity name function
• [ - token jade inline entity name function
• span - token jade inline name tag meta other entity
• %20 - token jade inline
• text - token jade inline name tag meta other entity
• . - token constant language js jade inline tag
• ] - token constant language js jade inline tag invalid illegal
• Two - token jade inline name tag meta other entity
• %20 - token jade inline
• text - token jade inline name tag meta other entity
• . - token constant language js jade inline tag

Highlighting

Expected result

Tokens

• .class - token constant language js
• %20One%20 - token
• # - token jade inline entity name function
• [ - token jade inline entity name function
• span - token jade inline name tag meta other entity
• text. - token
• ] - token name jade entity inline function
• %20Two text. - token

Let me know if I can help you with understanding of this problem.

[alexdima]

@mrmlnc

Most interesting, when I load only the jade grammar I get the desired results.

I think it has something to do with including also the JavaScript grammar. I will try that next.

(I have an inspect script in this project that helps with debugging)
npm run inspect ..\vscode-jade-syntax\syntaxes\jade.json input.txt >out.txt

> vscode-textmate@2.1.1 inspect c:\Alex\src\vscode-textmate
> node scripts/inspect.js "..\vscode-jade-syntax\syntaxes\jade.json" "input.txt"

LOADING GRAMMAR..\vscode-jade-syntax\syntaxes\jade.json


===========================================
TOKENIZING LINE 1: |.class One #[span text.] Two text.|

@@scanNext: |.class One #[span text.] Two text.\n|
  pushing BeginEndRule#105 @ jade.json:281 - ^\s*

@@scanNext: |.class One #[span text.] Two text.\n|
  pushing BeginEndRule#31 @ jade.json:443 - (?=[\w.#])|(:\s*)

@@scanNext: |.class One #[span text.] Two text.\n|
  matched MatchRule#53 @ jade.json:878 - (\.[a-zA-Z0-9_-]+)
  token: |.class|
      * text.jade
      * meta.selector.css.jade
      * entity.other.attribute-name.class.css.jade

@@scanNext: | One #[span text.] Two text.\n|
  pushing BeginEndRule#65 @ jade.json:961 - (?=.)

@@scanNext: | One #[span text.] Two text.\n|
  token: | One |
      * text.jade
  pushing BeginEndRule#66 @ jade.json:548 - (?<!\\)(#\[)
  token: |#[|
      * text.jade
      * inline.jade
      * entity.name.function.jade

@@scanNext: |span text.] Two text.\n|
  pushing BeginEndRule#71 @ jade.json:567 - (?<!\])(?=[\w.#])|(:\s*)

@@scanNext: |span text.] Two text.\n|
  pushing BeginEndRule#72 @ jade.json:936 - (?<=#\[)

@@scanNext: |span text.] Two text.\n|
  matched MatchRule#73 @ jade.json:940 - ((\w|-|:(?!\s))+)
  token: |span|
      * text.jade
      * inline.jade
      * tag.inline.jade
      * meta.tag.other entity.name.tag.jade

@@scanNext: | text.] Two text.\n|
  popping BeginEndRule#72 @ jade.json:936 - (?=(?<!:)\s|\])

@@scanNext: | text.] Two text.\n|
  popping BeginEndRule#71 @ jade.json:567 - (?=\]|(:.)|=|\s)

@@scanNext: | text.] Two text.\n|
  token: | |
      * text.jade
      * inline.jade
  pushing BeginEndRule#71 @ jade.json:567 - (?<!\])(?=[\w.#])|(:\s*)

@@scanNext: |text.] Two text.\n|
  popping BeginEndRule#71 @ jade.json:567 - (?=\]|(:.)|=|\s)
  token: |text.|
      * text.jade
      * inline.jade
      * tag.inline.jade

@@scanNext: |] Two text.\n|
  popping BeginEndRule#66 @ jade.json:548 - (\])
  token: |]|
      * text.jade
      * inline.jade
      * entity.name.function.jade

@@scanNext: | Two text.\n|
  popping BeginEndRule#65 @ jade.json:961 - $
  token: | Two text.|
      * text.jade

@@scanNext: |\n|
  popping BeginEndRule#31 @ jade.json:443 - (\.?$)|(?=:.)

@@scanNext: |\n|
  popping BeginEndRule#105 @ jade.json:281 - $

@@scanNext: |\n|
  no more matches.
  token: |\n|
      * text.jade

@@LINE END RULE STACK CONTAINS 1 RULES:
  * IncludeOnlyRule#1 @ jade.json:1  -- [1,1] "text.jade"

[alexdima]

@aeschli @mrmlnc

If I overwrite vscode/extensions/jade/syntaxes/Jade.json and with vscode-jade-syntax/syntaxes/jade.json I get the following (correct I believe) coloring:

So at this point I don't think this is an issue inside vscode-textmate.

@aeschli Perhaps you need to update the Jade grammar inside vscode? [I am not sure why you moved this issue in this repo?]

Please note vscode/extensions/jade/syntaxes/Jade.json is quite different than vscode-jade-syntax/syntaxes/jade.json although it has the same uuid

[mrmlnc]

@alexandrudima, my repository (vscode-jade-syntax) contains my fixes for this problem. But why Sublime Text works without changes?

[alexdima]

@mrmlnc Sorry for not understanding.

You say "Clarification: that issue also reproduced in Atom with this grammar, but not Sublime Text 2 & 3.". but I honestly don't know which grammar you mean.

I took the one from vscode-jade-syntax/syntaxes/jade.json . Can you please tell me which grammar you'd like me to try?

[mrmlnc]

@alexandrudima, oh, sorry, I talk about original Jade syntax:

https://github.com/Microsoft/vscode/tree/master/extensions/jade/syntaxes

[alexdima]

@mrmlnc Thanks for the clarification. So I'm using the grammar at https://github.com/Microsoft/vscode/blob/9ae0f9d05a7d0c4f8338f4433aee21ac0140485f/extensions/jade/syntaxes/Jade.json

I cannot speculate about Sublime's implementation (it could have its own bugs?), of interest to me would be the behaviour of TextMate. I don't have access to a mac right now, @aeschli can you please try our Jade grammar on this input in TextMate?

I have added even more debugging information and I have looked at the detailed debug output and ... I don't see any bug, it appears to me the engine interprets the syntax correctly. .

Here is my interpretation:

• relevant text is #[span text.]
• after #[, the engine is in the inline_jade rule (BeginEndRule#63 @ Jade.json:537)
• it then has to continue with span text.]
  • it enters the rule with the name tag.inline.jade (BeginEndRule#68 @ Jade.json:556)
    • it enters the tag_name rule (BeginEndRule#46 @ Jade.json:899) and consumes span
      • it then has to continue with text.]
    • it pops the tag_name rule (BeginEndRule#46 @ Jade.json:899)
  • it pops the rule with the name tag.inline.jade (BeginEndRule#68 @ Jade.json:556)
  • it enters the rule with the name tag.inline.jade (BeginEndRule#68 @ Jade.json:556) and consumes
    • it then has to continue with text.]
    • it enters the tag_name rule (BeginEndRule#46 @ Jade.json:899) and consumes text
      • it then has to continue with .]
    • it pops the tag_name rule (BeginEndRule#46 @ Jade.json:899)
  • it pops the rule with the name tag.inline.jade (BeginEndRule#68 @ Jade.json:556)
  • it then matches the tag_classes rule (MatchRule#50 @ Jade.json:867) which consumes . and ] and produces this constant.language.js and invalid.illegal.tag.jade

I don't know what the intent of the grammar is, is it correctly looking at this point for tag_classes or should it have done something different. From my point of view, without knowing the intentions of the grammars and without understanding in great detail the regular expressions involved, it appears to me the engine is taking the right decisions.

Here is the complete debug output:

> vscode-textmate@2.1.1 inspect c:\Alex\src\vscode-textmate
> node scripts/inspect.js "..\vscode\extensions\jade\syntaxes\Jade.json" "input.txt"

LOADING GRAMMAR..\vscode\extensions\jade\syntaxes\Jade.json


===========================================
TOKENIZING LINE 1: |.class One #[span text.] Two text.|

@@scanNext: |.class One #[span text.] Two text.\n|
  scanning for
   - 2: ^(!!!|doctype)(\s*[a-zA-Z0-9-_]+)?
   - 3: ^(\s*)//-
   - 4: ^(\s*)//
   - 8: ^(\s*)(script)(?=[.#(\s])((?![^\n]*type=)|(?=[^\n]*(text|application)/javascript))
   - 77: ^(\s*)(style)((\.$)|(?=[.#(].*\.$))
   - 81: ^(\s*):(sass)(?=\(|$)
   - 83: ^(\s*):(less)(?=\(|$)
   - 85: ^(\s*):(stylus)(?=\(|$)
   - 87: ^(\s*):(coffee(-?script)?)(?=\(|$)
   - 89: ^(\s*)((:(?=.))|(:$))
   - 95: ^(\s*)(?=[\w.#].*?\.$)(?=(?:(?:(?:(?:(?:#[\w-]+)|(?:\.[\w-]+))|(?:(?:[#!]\{[^}]*\})|(?:\w(?:(?:[\w:-]+[\w-])|(?:[\w-]*)))))(?:(?:#[\w-]+)|(?:\.[\w-]+)|(?:\((?:[^()\'\"]*(?:(?:\'(?:[^\']|(?:(?<!\\)\\\'))*\')|(?:\"(?:[^\"]|(?:(?<!\\)\\\"))*\")))*[^()]*\))*)*)(?:(?:(?::\s+)|(?<=\)))(?:(?:(?:(?:#[\w-]+)|(?:\.[\w-]+))|(?:(?:[#!]\{[^}]*\})|(?:\w(?:(?:[\w:-]+[\w-])|(?:[\w-]*)))))(?:(?:#[\w-]+)|(?:\.[\w-]+)|(?:\((?:[^()\'\"]*(?:(?:\'(?:[^\']|(?:(?<!\\)\\\'))*\')|(?:\"(?:[^\"]|(?:(?<!\\)\\\"))*\")))*[^()]*\))*)*))*)\.$)(?:(?:(#[\w-]+)|(\.[\w-]+))|((?:[#!]\{[^}]*\})|(?:\w(?:(?:[\w:-]+[\w-])|(?:[\w-]*)))))
   - 100: ^\s*
  pushing BeginEndRule#100 @ Jade.json:270 - ^\s*

@@scanNext: |.class One #[span text.] Two text.\n|
  scanning for
   - -1: $
   - 63: (?<!\\)(#\[)
   - 30: (extends|include|yield|append|prepend|block( (append|prepend))?)\s+(.*)$
   - 33: (-|(([a-zA-Z0-9_]+)\s+=))
   - 101: (mixin\s+)([\w-]+)(?:(\()\s*((?:[a-zA-Z_]\w*\s*)(?:,\s*[a-zA-Z_]\w*\s*)*)(\)))?$
   - 35: ((?:mixin\s+)|\+)([\w-]+)
   - 107: (default|when)((\s+|(?=:))|$)
   - 114: \|
   - 59: (!?\=)\s*
   - 115: \G(?=(#[^\{\w-])|[^\w.#])
   - 29: (?=[\w.#])|(:\s*)
  pushing BeginEndRule#29 @ Jade.json:432 - (?=[\w.#])|(:\s*)

@@scanNext: |.class One #[span text.] Two text.\n|
  scanning for
   - -1: (\.?$)|(?=:.)
   - 30: (extends|include|yield|append|prepend|block( (append|prepend))?)\s+(.*)$
   - 33: (-|(([a-zA-Z0-9_]+)\s+=))
   - 35: ((?:mixin\s+)|\+)([\w-]+)
   - 45: (?<=:)\w.*$
   - 46: ([#!]\{(?=.*?\}))|(\w(([\w:-]+[\w-])|([\w-]*)))
   - 49: #[\w-]+
   - 50: \.([^\w-])?[\w-]*
   - 11: (\(\s*)
   - 52: (&attributes\()
   - 56: ((\.)\s+$)|((:)\s*$)
   - 59: (!?\=)\s*
   - 62: (?=.)
  matched MatchRule#50 @ Jade.json:867 - \.([^\w-])?[\w-]*
  token: |.class|
      * text.jade
      * constant.language.js

@@scanNext: | One #[span text.] Two text.\n|
  scanning for
   - -1: (\.?$)|(?=:.)
   - 30: (extends|include|yield|append|prepend|block( (append|prepend))?)\s+(.*)$
   - 33: (-|(([a-zA-Z0-9_]+)\s+=))
   - 35: ((?:mixin\s+)|\+)([\w-]+)
   - 45: (?<=:)\w.*$
   - 46: ([#!]\{(?=.*?\}))|(\w(([\w:-]+[\w-])|([\w-]*)))
   - 49: #[\w-]+
   - 50: \.([^\w-])?[\w-]*
   - 11: (\(\s*)
   - 52: (&attributes\()
   - 56: ((\.)\s+$)|((:)\s*$)
   - 59: (!?\=)\s*
   - 62: (?=.)
  pushing BeginEndRule#62 @ Jade.json:920 - (?=.)

@@scanNext: | One #[span text.] Two text.\n|
  scanning for
   - -1: $
   - 63: (?<!\\)(#\[)
   - 73: (?=<[^>]*>)
   - 75: (&)([a-zA-Z0-9]+|#[0-9]+|#x[0-9a-fA-F]+)(;)
   - 76: [<>&]
   - 21: (?<!\\)[#!]\{(?=.*?\})
   - 23: (?<!\\)[#!]\{(?=[^}]*$)
  token: | One |
      * text.jade
  pushing BeginEndRule#63 @ Jade.json:537 - (?<!\\)(#\[)
  token: |#[|
      * text.jade
      * inline.jade
      * entity.name.function.jade

@@scanNext: |span text.] Two text.\n|
  scanning for
   - -1: (\])
   - 63: (?<!\\)(#\[)
   - 35: ((?:mixin\s+)|\+)([\w-]+)
   - 68: (?<!\])(?=[\w.#])|(:\s*)
   - 33: (-|(([a-zA-Z0-9_]+)\s+=))
   - 59: (!?\=)\s*
   - 70: \[
   - 72: \[
   - 63: (?<!\\)(#\[)
   - 73: (?=<[^>]*>)
   - 75: (&)([a-zA-Z0-9]+|#[0-9]+|#x[0-9a-fA-F]+)(;)
   - 76: [<>&]
   - 21: (?<!\\)[#!]\{(?=.*?\})
   - 23: (?<!\\)[#!]\{(?=[^}]*$)
  pushing BeginEndRule#68 @ Jade.json:556 - (?<!\])(?=[\w.#])|(:\s*)

@@scanNext: |span text.] Two text.\n|
  scanning for
   - -1: (?=\]|(:.)|=|\s)
   - 46: ([#!]\{(?=.*?\}))|(\w(([\w:-]+[\w-])|([\w-]*)))
   - 49: #[\w-]+
   - 50: \.([^\w-])?[\w-]*
   - 11: (\(\s*)
   - 52: (&attributes\()
   - 63: (?<!\\)(#\[)
   - 69: \[
  pushing BeginEndRule#46 @ Jade.json:899 - ([#!]\{(?=.*?\}))|(\w(([\w:-]+[\w-])|([\w-]*)))
  token: |span|
      * text.jade
      * inline.jade
      * tag.inline.jade
      * meta.tag.other entity.name.tag.jade

@@scanNext: | text.] Two text.\n|
  scanning for
   - -1: (\G(?<![^\w-]))|\}|$
   - 47: \G(?<=\{)
  popping BeginEndRule#46 @ Jade.json:899 - (\G(?<!\5[^\w-]))|\}|$

@@scanNext: | text.] Two text.\n|
  scanning for
   - -1: (?=\]|(:.)|=|\s)
   - 46: ([#!]\{(?=.*?\}))|(\w(([\w:-]+[\w-])|([\w-]*)))
   - 49: #[\w-]+
   - 50: \.([^\w-])?[\w-]*
   - 11: (\(\s*)
   - 52: (&attributes\()
   - 63: (?<!\\)(#\[)
   - 69: \[
  popping BeginEndRule#68 @ Jade.json:556 - (?=\]|(:.)|=|\s)

@@scanNext: | text.] Two text.\n|
  scanning for
   - -1: (\])
   - 63: (?<!\\)(#\[)
   - 35: ((?:mixin\s+)|\+)([\w-]+)
   - 68: (?<!\])(?=[\w.#])|(:\s*)
   - 33: (-|(([a-zA-Z0-9_]+)\s+=))
   - 59: (!?\=)\s*
   - 70: \[
   - 72: \[
   - 63: (?<!\\)(#\[)
   - 73: (?=<[^>]*>)
   - 75: (&)([a-zA-Z0-9]+|#[0-9]+|#x[0-9a-fA-F]+)(;)
   - 76: [<>&]
   - 21: (?<!\\)[#!]\{(?=.*?\})
   - 23: (?<!\\)[#!]\{(?=[^}]*$)
  token: | |
      * text.jade
      * inline.jade
  pushing BeginEndRule#68 @ Jade.json:556 - (?<!\])(?=[\w.#])|(:\s*)

@@scanNext: |text.] Two text.\n|
  scanning for
   - -1: (?=\]|(:.)|=|\s)
   - 46: ([#!]\{(?=.*?\}))|(\w(([\w:-]+[\w-])|([\w-]*)))
   - 49: #[\w-]+
   - 50: \.([^\w-])?[\w-]*
   - 11: (\(\s*)
   - 52: (&attributes\()
   - 63: (?<!\\)(#\[)
   - 69: \[
  pushing BeginEndRule#46 @ Jade.json:899 - ([#!]\{(?=.*?\}))|(\w(([\w:-]+[\w-])|([\w-]*)))
  token: |text|
      * text.jade
      * inline.jade
      * tag.inline.jade
      * meta.tag.other entity.name.tag.jade

@@scanNext: |.] Two text.\n|
  scanning for
   - -1: (\G(?<![^\w-]))|\}|$
   - 47: \G(?<=\{)
  popping BeginEndRule#46 @ Jade.json:899 - (\G(?<!\5[^\w-]))|\}|$

@@scanNext: |.] Two text.\n|
  scanning for
   - -1: (?=\]|(:.)|=|\s)
   - 46: ([#!]\{(?=.*?\}))|(\w(([\w:-]+[\w-])|([\w-]*)))
   - 49: #[\w-]+
   - 50: \.([^\w-])?[\w-]*
   - 11: (\(\s*)
   - 52: (&attributes\()
   - 63: (?<!\\)(#\[)
   - 69: \[
  matched MatchRule#50 @ Jade.json:867 - \.([^\w-])?[\w-]*
  token: |.|
      * text.jade
      * inline.jade
      * tag.inline.jade
      * constant.language.js
  token: |]|
      * text.jade
      * inline.jade
      * tag.inline.jade
      * constant.language.js
      * invalid.illegal.tag.jade

@@scanNext: | Two text.\n|
  scanning for
   - -1: (?=\]|(:.)|=|\s)
   - 46: ([#!]\{(?=.*?\}))|(\w(([\w:-]+[\w-])|([\w-]*)))
   - 49: #[\w-]+
   - 50: \.([^\w-])?[\w-]*
   - 11: (\(\s*)
   - 52: (&attributes\()
   - 63: (?<!\\)(#\[)
   - 69: \[
  popping BeginEndRule#68 @ Jade.json:556 - (?=\]|(:.)|=|\s)

@@scanNext: | Two text.\n|
  scanning for
   - -1: (\])
   - 63: (?<!\\)(#\[)
   - 35: ((?:mixin\s+)|\+)([\w-]+)
   - 68: (?<!\])(?=[\w.#])|(:\s*)
   - 33: (-|(([a-zA-Z0-9_]+)\s+=))
   - 59: (!?\=)\s*
   - 70: \[
   - 72: \[
   - 63: (?<!\\)(#\[)
   - 73: (?=<[^>]*>)
   - 75: (&)([a-zA-Z0-9]+|#[0-9]+|#x[0-9a-fA-F]+)(;)
   - 76: [<>&]
   - 21: (?<!\\)[#!]\{(?=.*?\})
   - 23: (?<!\\)[#!]\{(?=[^}]*$)
  token: | |
      * text.jade
      * inline.jade
  pushing BeginEndRule#68 @ Jade.json:556 - (?<!\])(?=[\w.#])|(:\s*)

@@scanNext: |Two text.\n|
  scanning for
   - -1: (?=\]|(:.)|=|\s)
   - 46: ([#!]\{(?=.*?\}))|(\w(([\w:-]+[\w-])|([\w-]*)))
   - 49: #[\w-]+
   - 50: \.([^\w-])?[\w-]*
   - 11: (\(\s*)
   - 52: (&attributes\()
   - 63: (?<!\\)(#\[)
   - 69: \[
  pushing BeginEndRule#46 @ Jade.json:899 - ([#!]\{(?=.*?\}))|(\w(([\w:-]+[\w-])|([\w-]*)))
  token: |Two|
      * text.jade
      * inline.jade
      * tag.inline.jade
      * meta.tag.other entity.name.tag.jade

@@scanNext: | text.\n|
  scanning for
   - -1: (\G(?<![^\w-]))|\}|$
   - 47: \G(?<=\{)
  popping BeginEndRule#46 @ Jade.json:899 - (\G(?<!\5[^\w-]))|\}|$

@@scanNext: | text.\n|
  scanning for
   - -1: (?=\]|(:.)|=|\s)
   - 46: ([#!]\{(?=.*?\}))|(\w(([\w:-]+[\w-])|([\w-]*)))
   - 49: #[\w-]+
   - 50: \.([^\w-])?[\w-]*
   - 11: (\(\s*)
   - 52: (&attributes\()
   - 63: (?<!\\)(#\[)
   - 69: \[
  popping BeginEndRule#68 @ Jade.json:556 - (?=\]|(:.)|=|\s)

@@scanNext: | text.\n|
  scanning for
   - -1: (\])
   - 63: (?<!\\)(#\[)
   - 35: ((?:mixin\s+)|\+)([\w-]+)
   - 68: (?<!\])(?=[\w.#])|(:\s*)
   - 33: (-|(([a-zA-Z0-9_]+)\s+=))
   - 59: (!?\=)\s*
   - 70: \[
   - 72: \[
   - 63: (?<!\\)(#\[)
   - 73: (?=<[^>]*>)
   - 75: (&)([a-zA-Z0-9]+|#[0-9]+|#x[0-9a-fA-F]+)(;)
   - 76: [<>&]
   - 21: (?<!\\)[#!]\{(?=.*?\})
   - 23: (?<!\\)[#!]\{(?=[^}]*$)
  token: | |
      * text.jade
      * inline.jade
  pushing BeginEndRule#68 @ Jade.json:556 - (?<!\])(?=[\w.#])|(:\s*)

@@scanNext: |text.\n|
  scanning for
   - -1: (?=\]|(:.)|=|\s)
   - 46: ([#!]\{(?=.*?\}))|(\w(([\w:-]+[\w-])|([\w-]*)))
   - 49: #[\w-]+
   - 50: \.([^\w-])?[\w-]*
   - 11: (\(\s*)
   - 52: (&attributes\()
   - 63: (?<!\\)(#\[)
   - 69: \[
  pushing BeginEndRule#46 @ Jade.json:899 - ([#!]\{(?=.*?\}))|(\w(([\w:-]+[\w-])|([\w-]*)))
  token: |text|
      * text.jade
      * inline.jade
      * tag.inline.jade
      * meta.tag.other entity.name.tag.jade

@@scanNext: |.\n|
  scanning for
   - -1: (\G(?<![^\w-]))|\}|$
   - 47: \G(?<=\{)
  popping BeginEndRule#46 @ Jade.json:899 - (\G(?<!\5[^\w-]))|\}|$

@@scanNext: |.\n|
  scanning for
   - -1: (?=\]|(:.)|=|\s)
   - 46: ([#!]\{(?=.*?\}))|(\w(([\w:-]+[\w-])|([\w-]*)))
   - 49: #[\w-]+
   - 50: \.([^\w-])?[\w-]*
   - 11: (\(\s*)
   - 52: (&attributes\()
   - 63: (?<!\\)(#\[)
   - 69: \[
  matched MatchRule#50 @ Jade.json:867 - \.([^\w-])?[\w-]*
  token: |.|
      * text.jade
      * inline.jade
      * tag.inline.jade
      * constant.language.js
  token: |\n|
      * text.jade
      * inline.jade
      * tag.inline.jade
      * constant.language.js
      * invalid.illegal.tag.jade

@@scanNext: |n|
  scanning for
   - -1: (?=\]|(:.)|=|\s)
   - 46: ([#!]\{(?=.*?\}))|(\w(([\w:-]+[\w-])|([\w-]*)))
   - 49: #[\w-]+
   - 50: \.([^\w-])?[\w-]*
   - 11: (\(\s*)
   - 52: (&attributes\()
   - 63: (?<!\\)(#\[)
   - 69: \[
  no more matches.

@@LINE END RULE STACK CONTAINS 6 RULES:
  * IncludeOnlyRule#1 @ Jade.json:1  -- [1,6] "text.jade"
  * BeginEndRule#100 @ Jade.json:270  -- [100,5] "null"
  * BeginEndRule#29 @ Jade.json:432  -- [29,4] "null"
  * BeginEndRule#62 @ Jade.json:920  -- [62,3] "null"
  * BeginEndRule#63 @ Jade.json:537  -- [63,2] "inline.jade"
  * BeginEndRule#68 @ Jade.json:556  -- [68,1] "tag.inline.jade"

[alexdima]

@mrmlnc Any thoughts on this?

[mrmlnc]

Sorry, @alexandrudima, here should be just inline.jade:

@@scanNext: |text.] Two text.\n|
  scanning for
   - -1: (?=\]|(:.)|=|\s)
   - 46: ([#!]\{(?=.*?\}))|(\w(([\w:-]+[\w-])|([\w-]*)))
   - 49: #[\w-]+
   - 50: \.([^\w-])?[\w-]*
   - 11: (\(\s*)
   - 52: (&attributes\()
   - 63: (?<!\\)(#\[)
   - 69: \[
  pushing BeginEndRule#46 @ Jade.json:899 - ([#!]\{(?=.*?\}))|(\w(([\w:-]+[\w-])|([\w-]*)))
  token: |text|
      * text.jade
      * inline.jade
      * tag.inline.jade
      * meta.tag.other entity.name.tag.jade

@@scanNext: |.] Two text.\n|
  scanning for
   - -1: (\G(?<![^\w-]))|\}|$
   - 47: \G(?<=\{)
  popping BeginEndRule#46 @ Jade.json:899 - (\G(?<!\5[^\w-]))|\}|$

Soon I'll see how work the regular expression from my repository. And let you know about my thoughts.

[mrmlnc]

Hello, @alexandrudima and @aeschli,

I think we can close this issue? I tried to work with TextMate2 and there's this grammar is working properly.

I see that you started updating some grammar (microsoft/vscode@f1672e4, microsoft/vscode@fbdf234 and etc.).

Well, I think I'll be contribute to one of the implementations of Jade grammar and try to solve all my problems. Later I'll create the issue in VS Code repository with request to update Jade grammar.

[msftrncs]

This is an old issue, but PR #86 would fix this. The error occurs at:

@@scanNext: | Two text.\n|
  scanning for
   - -1: (\])
   - 63: (?<!\\)(#\[)
   - 35: ((?:mixin\s+)|\+)([\w-]+)
   - 68: (?<!\])(?=[\w.#])|(:\s*)
   - 33: (-|(([a-zA-Z0-9_]+)\s+=))
   - 59: (!?\=)\s*
   - 70: \[
   - 72: \[
   - 63: (?<!\\)(#\[)
   - 73: (?=<[^>]*>)
   - 75: (&)([a-zA-Z0-9]+|#[0-9]+|#x[0-9a-fA-F]+)(;)
   - 76: [<>&]
   - 21: (?<!\\)[#!]\{(?=.*?\})
   - 23: (?<!\\)[#!]\{(?=[^}]*$)
  token: | |
      * text.jade
      * inline.jade
  pushing BeginEndRule#68 @ Jade.json:556 - (?<!\])(?=[\w.#])|(:\s*)

See how items 70 and 72 are the same? That is happening because a rule is getting skipped because its begin value is "" (empty), Item 72 is the first of its inner patterns, but because its blank begin rule is ignored, the other rules are not pushed away, and so the tag rule just keeps consuming the text (matching rule 68), until the end rule for the closing bracket is met.

The detail of two identical rules is what tipped me to look at the grammar closer, to catch the empty begin.