Allow multiline comments for text cells in the py:percent format

[hpmsi1]

Thank you for your very useful program jupytext.

I am trying to convert a markdown section in .py file to .ipynb. It works but produces extra """ and beginning and end of the cell. Example given below:

#%% [markdown]
'''

$$ \begin{align} \dot{x} & = \sigma(y-x) \\ \dot{y} & = \rho x - y - xz \\ \dot{z} & = -\beta z + xy \end{align} $$

'''

Is it possible that these """ don't appear in the converted ipynb cell?

Thank you

[mwouts]

Hello @hpmsi1 , well, yes and no. Currently the Markdown cell should only use line comments, i.e. your input should be

# %% [markdown]
# $$
# \begin{align}
# \dot{x} & = \sigma(y-x) \
# \dot{y} & = \rho x - y - xz \
# \dot{z} & = -\beta z + xy
# \end{align}
# $$

We could think of implementing support for multiline strings - but I won't be available for that before a few weeks.

[hpmsi1]

Thank you for responding so quickly.
I will use your solution above for now. Eventually it will be very useful to be able to embed large markdown blocks as I intend to develop tutorials for students and this feature will be very useful.
Kind regards,
Mohammad

[mwouts]

You're welcome! Yes we will do that in the future.

By the way, if you are looking for a way to embed large markdown blocks, I suggest you open and edit your .py document as a notebook directly in Jupyter. There you will be able to edit or paste the markdown content easily, and also you will find out how the format works.

Note also that if you plan to input a lot of Markdown content, maybe the Markdown format is a better fit for your notebooks... (use Jupytext at the command line to convert from one formats to the other).

[hpmsi1]

Thank you for the suggestions: I am writing all code in .py as these are easier to maintain, develop and debug.
Then I use Jupytext to create ipynb. Then I use nbsphinx to create ready to distribute code and documentation. I will write a bash script to automate the whole process. So your jupytext is very instrumental in the pipeline.

[mwouts]

Oh that's interesting, actually you're implementing one example of #247. There's one comment there that may be of interest to you: nbsphinx can take Jupytext scripts directly as inputs.

[phaustin]

To follow up on my comment in #247 about scaled images --I found this pair of functions that can go back and forth between img tags (for ipynb) and gfm markdown (for pandoc rst). You can put them in a precommit hook to handle the translation needed for nbspninx

source:
https://github.com/jupyter/notebook/issues/1095

#I have exactly the problem of @amueller on the image size issue. I end up in writing a 
#simple script with the following functions to convert between ![](){width=..} and <img></img> syntax.

re_mdimg = re.compile(r"(!\[([^\]]*)\]\(([^\)]*)\)\{(.+?)\})")
re_imgtag = re.compile(r"(<img ([^>]*?)(?:/>|>(.*?)</img>))")


def mdimg_to_imgtag(para):
    for (match, tag, link, opts) in re_mdimg.findall(para):
        w = re.search(r"width\s*=\s*(\S*)", opts)
        h = re.search(r"height\s*=\s*(\S*)", opts)
        width = "width: %s;" % w.group(1) if w else ""
        height = "height: %s;" % h.group(1) if h else ""
        style_opts = " style=\"%s%s\"" % (width, height) if w or h else ""

        if tag:
            para = para.replace(
                match,
                "<img src=\"%s\"%s>%s</img>" % (link, style_opts, tag))
        else:
            para = para.replace(
                match,
                "<img src=\"%s\"%s />" % (link, style_opts))
    return para


def imgtag_to_mdimg(para):
    for (match, opts, tag) in re_imgtag.findall(para):
        tag = tag if tag else ""
        l = re.search(r"""src\s*=\s*["'](\S*?)["']""", opts)
        w = re.search(r"""width\s*[=:]\s*["']?(\S+?)["'; ]""", opts)
        h = re.search(r"""height\s*[=:]\s*["']?(\S+?)["'; ]""", opts)
        style_opts = ""
        style_opts = "width=%s" % w.group(1) if w else ""
        if h:
            if style_opts:
                style_opts = style_opts + " "
            style_opts = style_opts + "height=%s" % h.group(1)
        para = para.replace(
            match,
            "![%s](%s){%s}" % (tag, l.group(1), style_opts))
    return para

[hpmsi1]

Thank you, this is really useful. I will follow up on this.

[mwouts]

@hpmsi1 , would you like to give a try to the new RC? Markdown cells defined with triple quotes should work now, see

jupytext/tests/test_read_simple_percent.py

Lines 212 to 225 in bf6049b

	def test_multiline_comments_in_markdown_1():
	text = """# %% [markdown]
	'''
	a
	long
	cell
	'''
	"""
	nb = jupytext.reads(text, 'py')
	assert len(nb.cells) == 1
	assert nb.cells[0].cell_type == 'markdown'
	assert nb.cells[0].source == "a\nlong\ncell"
	py = jupytext.writes(nb, 'py')
	compare(py, text)

The RC is available on pypi:

pip install jupytext==1.3.0rc0

[hpmsi1]

Great! Thank you so much for this upgrade. I will give it a try and let you know. Kind regards, Mohammad

…

On Sat, Oct 12, 2019 at 8:22 PM Marc Wouts ***@***.***> wrote: @hpmsi1 <https://github.com/hpmsi1> , would you like to give a try to the new RC? Markdown cells defined with triple quotes should work now, see https://github.com/mwouts/jupytext/blob/bf6049b9107fef171647345f84364cdb4c5aaf5e/tests/test_read_simple_percent.py#L212-L225 The RC is available on pypi: pip install jupytext==1.3.0rc0 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#305?email_source=notifications&email_token=ABVIKDXQ366WPZLYBKFVWGDQOJZ53A5CNFSM4IGRBJR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBCL6DA#issuecomment-541376268>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABVIKDSAVLNB5CC2RMC4KKDQOJZ53ANCNFSM4IGRBJRQ> .

[hpmsi1]

Thank you it works.
One minor new change in the output, thought you should know:
This code produces no line breaks anymore as it used to:

$$
\begin{align}
\dot{x} & = \sigma(y-x)\ 
\dot{y} & = \rho x - y - xz \ 
\dot{z} & = -\beta z + xy 
\end{align} 
$$

output:
𝑥˙=𝜎(𝑦−𝑥) 𝑦˙=𝜌𝑥−𝑦−𝑥𝑧 𝑧˙=−𝛽𝑧+𝑥𝑦

However, this line break issue is fixed by doing this:

$$
\begin{align}
\dot{x} & = \sigma(y-x)\\ 
\dot{y} & = \rho x - y - xz \\ 
\dot{z} & = -\beta z + xy 
\end{align} 
$$

output:

[mwouts]

Oh, that's interesting! A line break in tex is \\ (double backslash), but maybe it should be represented as four backslash when inside a Python string? And maybe \dot is not legitimate in a Python string, it should be \\dot? I'll have a look!

[mwouts]

@hpmsi1 , in the current implementation Jupytext takes the content of the markdown cell verbatim from the string. So the latex line break is simply \\, as in the markdown cell (by the way, I am not sure to see the difference between the two inputs in your example?)

[hpmsi1]

Thanks, I will use two slashes for line breaks.

(by the way, I am not sure to see the difference between the two inputs in your example?

My comment in edit mode shows the equations ending with two slash. But it changes it to a single slash when I submit it.

[mwouts]

Oh I see! I've edited your comment and used code markers (```) to preserve the raw text. I'm still a bit puzzle by the line return with a single slash, I'll do a few more experiments about that when time permits, and add the corresponding tests to Jupytext to make sure we understand this well.

[hpmsi1]

Thank you, very much appreciated.

…

On Tue, Oct 22, 2019 at 2:27 PM Marc Wouts ***@***.***> wrote: Oh I see! I've edited your comment and used code markers (```) to preserve the raw text. I'm still a bit puzzle by the line return with a single slash, I'll do a few more experiments about that when time permits, and add the corresponding tests to Jupytext to make sure we understand this well. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#305?email_source=notifications&email_token=ABVIKDT3UP3PPVCBZUYDK5DQP5H3PA5CNFSM4IGRBJR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEB65B7Y#issuecomment-545116415>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABVIKDSG4X7NFOAXVW7KL6TQP5H3PANCNFSM4IGRBJRQ> .

[mwouts]

Hello @hpmsi1 , I have added a test notebook, and made sure that round trips are consistent, whatever is in the markdown cell.

Just to confirm, I found no way to trigger a line break in the LaTeX equation with a single slash. Two backslash seem to be necessary:

The notebook is at tests/notebooks/ipynb_py/Line_breaks_in_LateX_305.ipynb, and its various representations are in the mirror folder, for instance:

jupytext/tests/notebooks/mirror/ipynb_to_percent/Line_breaks_in_LateX_305.py

Lines 31 to 42 in 3ef0e73

	# %% [markdown]
	'''
	This cell uses the triple quote cell markers introduced at https://github.com/mwouts/jupytext/issues/305
	$$
	\begin{align}
	\dot{x} & = \sigma(y-x)\\
	\dot{y} & = \rho x - y - xz \\
	\dot{z} & = -\beta z + xy
	\end{align}
	$$
	'''