md-math-block uses non-Oniguruma regex

[jensli]

Does this issue occur when all extensions are disabled?: No. This is an issue in the Markdown Math extension.

• VS Code Version: master branch commit 60fe2d5
• OS Version: NA

The following regex in md-math-block.tmLanguage.json is not a valid TextMate regex:

vscode/extensions/markdown-math/syntaxes/md-math-block.tmLanguage.json

Line 16 in 60fe2d5

"begin": "(?<=^\\s*)(\\${2})(?![^$]*\\${2})",

TextMate grammars should use the Oniguruma regex dialect. This is the regex engine used by Ruby. The given regex give the following result when tried in Ruby:

(?<=^\\s*)(\\${2})(?![^$]*\\${2})
=> jdoodle.rb:1: invalid pattern in look-behind: /(?<=^\\s*)(\\${2})(?![^$]*\\${2})/

Tested at jdoodle.com using Ruby 3.0.2.

Either the regex should be updated, or the regex implementation should be fixed to be compliant with Oniguruma.

About TextMate regexes:
https://macromates.com/manual/en/regular_expressions
https://code.visualstudio.com/api/language-extensions/syntax-highlight-guide#textmate-grammars

This issue was first observed here: eclipse-tm4e/tm4e#525

[vscodenpa]

Thanks for creating this issue! It looks like you may be using an old version of VS Code, the latest stable release is 1.78.0. Please try upgrading to the latest version and checking whether this issue remains.

Happy Coding!

[dcourv]

The issue appears to be that the first portion of the regex, (?<=^\\s*), is a lookbehind on a variable-length subexpression. Onigurama only supports lookbehind on a fixed-length subexpression.

[RedCMD]

vscodes TextMate engine uses the Oniguruma dialect
which does support variable length lookbehinds
https://github.com/kkos/oniguruma/blob/master/doc/SYNTAX.md#6-onig_syn_different_len_alt_look_behind-allow-abc

Oniguruma can handle either form, but not all regex engines can, so for compatibility, Oniguruma allows you to cause regexes for other regex engines to fail if they might depend on this rule.

look-behind length is however limited to 65536 characters

I personally use variable length look-behinds all the time in vscode (including negative-look-behinds)

[mjbvz]

Closing as this doesn't seem like an issue for how we're using the grammar. PRs welcome if you want to provide a reasonable fix for other users