Skip to content

Commit

Permalink
test: improve snapshot testing for js engine (#866)
Browse files Browse the repository at this point in the history 
Co-authored-by: Anthony Fu <github@antfu.me>
  • Loading branch information
@slevithan @antfu
slevithan and antfu authored Dec 17, 2024
1 parent 6d4b81c commit c489859
Show file tree
Hide file tree
Showing 35 changed files with 44,311 additions and 10,359 deletions.
35 changes: 19 additions & 16 deletions docs/references/engine-js-compat.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,19 @@

Compatibility reference of all built-in grammars with the [JavaScript RegExp engine](/guide/regex-engines#javascript-regexp-engine-experimental).

> Generated on Wednesday, November 27, 2024
> Generated on Monday, December 16, 2024
>
> Version `1.23.1`
> Version `1.24.2`
>
> Runtime: Node.js v22.11.0
## Report Summary

| | Count |
| :-------------- | --------------------------: |
| Total Languages | 215 |
| Supported | [197](#supported-languages) |
| Mismatched | [12](#mismatched-languages) |
| Total Languages | 218 |
| Supported | [199](#supported-languages) |
| Mismatched | [13](#mismatched-languages) |
| Unsupported | [6](#unsupported-languages) |

## Supported Languages
Expand Down Expand Up @@ -43,9 +43,10 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
| bibtex | ✅ OK | 19 | - | |
| bicep | ✅ OK | 27 | - | |
| blade | ✅ OK | 1126 | - | |
| bsl | ✅ OK | 96 | - | |
| c | ✅ OK | 177 | - | |
| cadence | ✅ OK | 71 | - | |
| cairo | ✅ OK | 80 | - | |
| cairo | ✅ OK | 236 | - | |
| clarity | ✅ OK | 43 | - | |
| clj | ✅ OK | 38 | - | |
| clojure | ✅ OK | 38 | - | |
Expand All @@ -55,14 +56,14 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
| codeql | ✅ OK | 151 | - | |
| coffee | ✅ OK | 469 | - | |
| common-lisp | ✅ OK | 60 | - | |
| coq | ✅ OK | 25 | - | |
| coq | ✅ OK | 26 | - | |
| crystal | ✅ OK | 1067 | - | |
| css | ✅ OK | 141 | - | |
| csv | ✅ OK | 1 | - | |
| cue | ✅ OK | 85 | - | |
| cypher | ✅ OK | 39 | - | |
| d | ✅ OK | 270 | - | |
| dart | ✅ OK | 72 | - | |
| dart | ✅ OK | 76 | - | |
| dax | ✅ OK | 23 | - | |
| desktop | ✅ OK | 16 | - | |
| diff | ✅ OK | 16 | - | |
Expand All @@ -82,8 +83,8 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
| fortran-free-form | ✅ OK | 328 | - | |
| fsharp | ✅ OK | 239 | - | |
| fsl | ✅ OK | 30 | - | |
| gdresource | ✅ OK | 157 | - | |
| gdscript | ✅ OK | 93 | - | |
| gdresource | ✅ OK | 159 | - | |
| gdscript | ✅ OK | 95 | - | |
| gdshader | ✅ OK | 39 | - | |
| genie | ✅ OK | 20 | - | |
| gherkin | ✅ OK | 19 | - | |
Expand All @@ -92,7 +93,7 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
| glimmer-ts | ✅ OK | 676 | - | |
| glsl | ✅ OK | 186 | - | |
| gnuplot | ✅ OK | 82 | - | |
| go | ✅ OK | 123 | - | |
| go | ✅ OK | 125 | - | |
| graphql | ✅ OK | 448 | - | |
| groovy | ✅ OK | 133 | - | |
| haml | ✅ OK | 562 | - | |
Expand Down Expand Up @@ -122,17 +123,16 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
| latex | ✅ OK | 283 | - | |
| lean | ✅ OK | 32 | - | |
| less | ✅ OK | 280 | - | |
| liquid | ✅ OK | 684 | - | |
| liquid | ✅ OK | 689 | - | |
| log | ✅ OK | 30 | - | |
| logo | ✅ OK | 9 | - | |
| lua | ✅ OK | 278 | - | |
| luau | ✅ OK | 88 | - | |
| make | ✅ OK | 51 | - | |
| marko | ✅ OK | 926 | - | |
| matlab | ✅ OK | 88 | - | |
| mdc | ✅ OK | 783 | - | |
| mdx | ✅ OK | 197 | - | |
| mermaid | ✅ OK | 129 | - | |
| mermaid | ✅ OK | 134 | - | |
| mipsasm | ✅ OK | 17 | - | |
| mojo | ✅ OK | 213 | - | |
| move | ✅ OK | 120 | - | |
Expand All @@ -159,7 +159,7 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
| qmldir | ✅ OK | 7 | - | |
| qss | ✅ OK | 31 | - | |
| r | ✅ OK | 71 | - | |
| racket | ✅ OK | 69 | - | |
| racket | ✅ OK | 69 | - | 8 |
| raku | ✅ OK | 52 | - | |
| reg | ✅ OK | 9 | - | |
| regexp | ✅ OK | 34 | - | |
Expand All @@ -172,6 +172,7 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
| scala | ✅ OK | 117 | - | |
| scheme | ✅ OK | 34 | - | |
| scss | ✅ OK | 234 | - | |
| sdbl | ✅ OK | 23 | - | |
| shaderlab | ✅ OK | 87 | - | |
| shellscript | ✅ OK | 148 | - | |
| shellsession | ✅ OK | 150 | - | |
Expand All @@ -187,9 +188,10 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
| svelte | ✅ OK | 637 | - | |
| system-verilog | ✅ OK | 102 | - | |
| systemd | ✅ OK | 32 | - | |
| talonscript | ✅ OK | 44 | - | |
| tasl | ✅ OK | 23 | - | |
| tcl | ✅ OK | 34 | - | |
| templ | ✅ OK | 673 | - | |
| templ | ✅ OK | 675 | - | |
| terraform | ✅ OK | 68 | - | |
| tex | ✅ OK | 106 | - | |
| toml | ✅ OK | 44 | - | |
Expand Down Expand Up @@ -242,6 +244,7 @@ Languages that do not throw with the JavaScript RegExp engine, but will produce
| kotlin | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=kotlin) | 58 | - | 1953 |
| kusto | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=kusto) | 60 | - | 40 |
| markdown | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=markdown) | 118 | - | 915 |
| mdc | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=mdc) | 783 | - | 702 |
| nginx | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=nginx) | 378 | - | 4 |
| php | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=php) | 1131 | - | 605 |
| po | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=po) | 23 | - | 423 |
Expand Down
1 change: 1 addition & 0 deletions package.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@
"mdast-util-gfm": "catalog:",
"mdast-util-to-hast": "catalog:",
"ofetch": "catalog:",
"ohash": "catalog:",
"picocolors": "catalog:",
"pnpm": "catalog:",
"prettier": "catalog:",
Expand Down
10 changes: 7 additions & 3 deletions packages/engine-javascript/src/index.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,11 @@ export function defaultJavaScriptRegexConstructor(pattern: string, options?: Oni
hasIndices: true,
rules: {
allowOrphanBackrefs: true,
// Dropping unhandled `\G` anchors without erroring allows more grammars to pass but also
// allows some false positive and negative matches. It's mostly because this is true that
// the JS engine is marked as experimental
allowUnhandledGAnchors: true,
// Improves search performance for generated regexes
asciiWordBoundaries: true,
},
...options,
Expand Down Expand Up @@ -106,7 +110,7 @@ export class JavaScriptScanner implements PatternScanner {
})
}

findNextMatchSync(string: string | RegexEngineString, startPosition: number): IOnigMatch | null {
findNextMatchSync(string: string | RegexEngineString, startPosition: number, _options: number): IOnigMatch | null {
const str = typeof string === 'string'
? string
: string.content
Expand All @@ -118,15 +122,15 @@ export class JavaScriptScanner implements PatternScanner {
captureIndices: match.indices!.map((indice) => {
if (indice == null) {
return {
end: MAX,
start: MAX,
end: MAX,
length: 0,
}
}
return {
start: indice[0] + offset,
length: indice[1] - indice[0],
end: indice[1] + offset,
length: indice[1] - indice[0],
}
}),
}
Expand Down
Loading

0 comments on commit c489859

Please sign in to comment.