Syntax highlight doesn't support fenced code block

[mitya57]

Although the parser supports fenced code block, the syntax highlighter seems not.

e.g.

```
# this should not be headered
```

Reported by: colinfang

Original Ticket: retext/tickets/98

[mitya57]

Confirmed, though it would require serious highlighter refactoring since it only operates on individual lines currently.

Original comment by: mandriver

[emanuel-th]

I saw #253 and thought fenced code blocks are not supported by markdown, only by commonmark? The original markdown page does not mention it.

Anyway I have a more complex example. A fenced code block within a list.
see list-code.md
The first has the problem as described in #158. The second renders correctly, but each list item has number one. As the block code is not indented and it starts a new list?

I'm using retext 7.2.3.

If you paste it as comment, Github renders it as intended.

[mitya57]

Hi @emanuel-th!

I saw #253 and thought fenced code blocks are not supported by markdown, only by commonmark? The original markdown page does not mention it.

The fenced code blocks are not part of the standard Markdown syntax, however Python-Markdown supports them via the Fenced Code Blocks extension. And in ReText, the Extra extension set (which includes fenced_code) is enabled by default.

Anyway I have a more complex example. A fenced code block within a list. see list-code.md The first has the problem as described in #158.

As the documentation of the Fenced Code extension says,

Warning

Fenced Code Blocks are only supported at the document root level. Therefore, they cannot be nested inside lists or blockquotes. If you need to nest fenced code blocks, you may want to try the third party extension SuperFences instead.

So, indeed, SuperFences should solve the problem of using the fenced code block in a list. See our wiki for details on how to enable extensions.

The second renders correctly, but each list item has number one. As the block code is not indented and it starts a new list?

Yes, in order for something to be appended to a list item, it needs to be indented with 4 spaces. Otherwise it is not part of the list.

Alternatively, you can use normal code blocks. For that, you need 8 spaces indentation (first 4 for a list item and another 4 for a code block). So, this will render as you expect:

1. some bash file

        #!/bin/bash
        echo "Hello World!"

2. second headline

[emanuel-th]

I'd prefer 4 space indent. But I have to convince all colleagues first. :)

I don't get superfences to work. I installed the extention with pip3. In python console I can "import pymdownx.superfences" without problems.
I added line to $HOME/.config/markdown-extensions.txt and tried <!-- Required extensions: pymdownx.superfences --> at the beginning of the document, the rendering is still broken.

Any way to debug this?
strace shows it does not open $HOME/.config/markdown-extensions.txt.
It access several files under $HOME/.local/ where the markdown extensions are installed, superfences.py is not one of them.

[mitya57]

There was a bug in interaction between mdx_posmap (used for synchronized scrolling) and superfences in lists.

I fixed it in master branch, commit 8bd8f58. Any chance you could try the version from master?

[emanuel-th]

Tested it, still produces a heading.

[mitya57]

Can you paste or attach your Markdown file, please?

[emanuel-th]

Same as above.
Also tested with
<!-- Required extensions: pymdownx.superfences -->
header.

[mitya57]

I mean, can you show me your Markdown source file? Either by attaching it here, or at least by providing a screenshot.

Ideally, please also attach the generated HTML (either by copying it from “View HTML code” dialog or by exporting to a file).

If the file cannot be shared, maybe you can try to make a minimal synthetic example that triggers this problem?

[emanuel-th]

list-code.md

html files: "We don’t support that file type." Using gist. When pasting html there it adds the empty lines.
https://gist.github.com/emanuel-th/8aa571c5576d2c75e9221cf761a94373

[mitya57]

Your input file produces the correct result with ReText version from master branch (e72f17c):

.

I notice that your HTML file has this line:

<meta name="generator" content="ReText 7.2.3">

which means you are using an older version. The version from master will identify itself as ReText 8.1.0 (Git).

Anyway, the next stable release will include this fix.

[emanuel-th]

I'm using a python env to run it. Maybe I did it wrong at some point.
Tried again, with the master version (8bd8f58) and (e72f17c):

It only works sometimes. I simple do: open retext, open the file, hit preview.
Then close retext and start again.
Sometimes it renders correctly, sometimes it does not.

[mitya57]

Right, I can reproduce this.

It looks like it depends on the order in which extensions are applied: both pymdownx.superfences and fenced_code try to provide handler for triple-backtick syntax, and whichever is applied last takes the priority.

To solve this, add the virtual remove_extra name to the list of extensions:

<!-- Required extensions: pymdownx.superfences, remove_extra -->

This way it should work always.

Also, I will try to make the order of extensions deterministic, to handle similar situations better.