Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Detect code blocks more leniently #155

Conversation

SomeoneToIgnore
Copy link

Fixes #135

@SomeoneToIgnore SomeoneToIgnore force-pushed the code-blocks-and-whitespaces branch from 29bd152 to 69ebd97 Compare July 22, 2024 20:05
@clason clason requested a review from MDeiml July 23, 2024 11:34
@@ -1988,8 +1988,10 @@ Example 117 - https://github.github.com/gfm/#example-117
(fenced_code_block
(fenced_code_block_delimiter)
(block_continuation)
(code_fence_content
(block_continuation))
(ERROR
Copy link
Collaborator

@clason clason Jul 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this ERROR expected? Shouldn't this be parsed as a fenced_code_block containing "``` aaa"?

Copy link
Author

@SomeoneToIgnore SomeoneToIgnore Jul 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea what this grammars expects.
GitHub seems to render the way you've mentioned indeed: https://gist.github.com/SomeoneToIgnore/6c87cec69e6aa29fac76cb8ac7230912

Do you have any ideas how to fix it properly?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, sorry. I'd suspect the scanner needs to lookahead further to the end of the line, and then check if there are any non-whitespace characters.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What complicates things (besides my diminishing knowledge of the area) is the fact that pure split_parser branch fails for me in many tests:

  extension_wikilink:
    ✗ Basic Wiki-link parsing.
    ✗ Wiki-link to a file
    ✗ Wiki-link to a heading in a note
    ✗ Wiki-link with title
    ✓ Wiki-link opener with no closer
    ✗ Wiki-link version of Example 556
  failing:
     Example 355 - https://github.github.com/gfm/#example-355
     Example 363 - https://github.github.com/gfm/#example-363
     Example 389 - https://github.github.com/gfm/#example-389
     Example 394 - https://github.github.com/gfm/#example-394
     Example 400 - https://github.github.com/gfm/#example-400
     Example 401 - https://github.github.com/gfm/#example-401
     Example 407 - https://github.github.com/gfm/#example-407
     Example 406 - https://github.github.com/gfm/#example-406
     Example 411 - https://github.github.com/gfm/#example-411
     Example 420 - https://github.github.com/gfm/#example-420
     Example 421 - https://github.github.com/gfm/#example-421
     Example 424 - https://github.github.com/gfm/#example-424
     Example 438 - https://github.github.com/gfm/#example-438
     Example 524 - https://github.github.com/gfm/#example-524
     Example 538 - https://github.github.com/gfm/#example-538
     Example 560 - https://github.github.com/gfm/#example-560
     Example 588 - https://github.github.com/gfm/#example-588
     Example 635 - https://github.github.com/gfm/#example-635

which does not simplify things.
So I'd love some guidance.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry that I did not answer in time. As clason mentioned, the parser should not output an error here. The problem is, that you do not "consume" the trailing whitespaces. There needs to be another while loop like the one above, but with whitespace instead of delimiter. (If you still want to work on this.)

I'm not sure why the tests fail for you. Try using the instructions here if you didn't already.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I didn't see there was another PR that fixed this problem already.

@ObserverOfTime ObserverOfTime force-pushed the split_parser branch 2 times, most recently from b7862ec to c25b635 Compare September 10, 2024 17:41
ribru17 added a commit to ribru17/tree-sitter-markdown that referenced this pull request Sep 28, 2024
@MDeiml
Copy link
Collaborator

MDeiml commented Oct 19, 2024

Superseeded by #163.

@MDeiml MDeiml closed this Oct 19, 2024
MDeiml pushed a commit that referenced this pull request Oct 19, 2024
* fix: allow whitespace after closing code block fence

Closes #135, supersedes #155
@SomeoneToIgnore SomeoneToIgnore deleted the code-blocks-and-whitespaces branch October 22, 2024 19:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Code block not recognized as one if there is trailing space on the closing fence
4 participants