triple backticks in a cell

[williamstein]

I remember once trying out Jupytext and the very first thing I tried was converting a notebook to markdown that contains a cell with three backticks in it, since I wanted to see how Jupytext solves this problem:

a = """
```
foo
```
"""

It turns out Jupytext doesn't solve this problem, and just produces a markdown file that looks like this

I think this is a bug.

I was reminded of this, because I just learned how to properly include three backticks in a fenced markdown code block. It's very clever. The fence is allowed to be any number $\geq 3$ of backticks in a row, so if you just happened to want to include say 4 backticks in a row inside a fenced code block, you instead use five backticks in a row for the fence like I'm doing right here in Github:

four backticks here: ````

I'm impressed with markdown. 🤷

In any case, perhaps it would be good to implement this in jupytext. Anything that correctly parses markdown will then work properly, and some unusually notebooks won't mysteriously break.

[mwouts]

Hi @williamstein , thanks for reporting this. Well that is a tricky case, isn't it? Already I find it not so bad that your example does not break Jupytext's round trip ability 😄

I have tested the later with

@pytest.mark.parametrize("fmt", ["py:light", "py:percent", "md", "md:myst"])
def test_triple_backticks_in_code_cell(fmt):
    nb = new_notebook(
        cells=[
            new_code_cell(
                '''a = """
```
foo
```
"""'''
            )
        ]
    )
    md = jupytext.writes(nb, fmt)
    nb2 = jupytext.reads(md, fmt)
    compare_notebooks(nb2, nb)

and it works on the first three formats. In md:myst we seem to lose the triple backticks in the value of a, I mean the cell becomes

a = """
foo
"""'''

I am not so sure that the markdown parser is easy to fix (could you please tell us what representation you'd expect for this notebook?). Or maybe it is - would you be fine if you get the option to tell Jupytext which cell boundary you'd like - above I have used ~~~?.

Aslo I was thinking that maybe the md:myst parser would be easier to evolve on this point. @chrisjsewell , what do you think about this use case? Is that something that you'd like to support in MyST?

[chrisjsewell]

yeh this sould be pretty easy to fix, just check for ` in the cell content, and increase the number of surrounding ` accordingly

[williamstein]

Yes exactly what @chrisjsewell says. The code I wrote in CoCalc looks like this:

...
        const info = node.info ?? "";
        // There is one special case with fenced codeblocks that we
        // have to worry about -- if they contain ```, then we need
        // to wrap with *more* than the max sequence of backticks
        // actually in the codeblock!   See
        //    https://stackoverflow.com/questions/49267811/how-can-i-escape-3-backticks-code-block-in-3-backticks-code-block
        // for an excellent discussion of this, and also
        // https://github.com/mwouts/jupytext/issues/712
        let fence = "```";
        while (value.indexOf(fence) != -1) {
          fence += "`";
        }
        return (
          fence + info + "\n" + ensure_ends_in_newline(value) + fence + "\n\n"
        );
...

It's slightly too aggressive, because it will use four backticks even if three backticks are NOT at the start of a line, but I think that's OK.

[mwouts]

Sure. Thanks for the suggestions, I'll give them a try.

[mwouts]

Hi @williamstein , the issue should be dealt with the merged PR.
Do you want to give a try to the development version?

pip install git+https://github.com/mwouts/jupytext.git

or

BUILD_JUPYTERLAB_EXTENSION=1 pip install git+https://github.com/mwouts/jupytext.git

if you want the extension for JupyterLab (requires nodejs).

By the way, please let us know if there is any other pattern that you would like to see supported!