Markdown output needs to escape some characters

[alerque]

I just tried to use git-cliff to update the changelog for a project (luacheck) that happens to have some commits that mention the LUA builtin global _ENV — without enclosing it in backticks. While it might have been better if I'd thought about this sooner and formatted it as code, generating a Markdown changlog is now a problem. I have to hand-edit the changelog to either escape the underscore (\_ENV) or add backticks (`_ENV`).

Given that it is also fair game to include things with the expectation that they will be Markdown I'm not sure how fixable this is. One idea I had is it might be possible to use a parser such as pulldown-cmark to check for unclosed markup and escape anything that causes tags to be left open ended. It might also be possible to just round trip the commit messages though that or another parser and get a variant that is "Markdown safe".

Note for example round tripping each commit message to pandoc would fix this problem:

$ echo "foo _ENV bar" | pandoc -t commonmark
foo \_ENV bar
$ echo "foo _ENV_ bar" | pandoc -t commonmark
foo *ENV* bar

[kaushalmodi]

Check out the new commit_preprocessors feature (see the README for examples).

You can use that to escape special Markdown characters (I use git-cliff to generate changelog files in Org mode. So I am glad that Markdown specific treatment is not baked into it!). Here's an untested example:

  commit_preprocessors = [
    { pattern = "[_]", replace = "\\${0}"}
  ]

If you wish, you can add other Markdown markup chars to the pattern like so: "[_*`]".

[orhun]

As @kaushalmodi said, using commit_preprocessors is the way to go for this. I'm not planning to add validation checks for different output formats including markdown.

[alerque]

It seems to be the commit_preprocessors is a bit of a misnomer. Having a regex replacement system available is handy for many uses cases, but it does not cover this use case at all. The naive example shown will blow up in all sorts of circumstances, including the simple MWE shown for the simple reason that in Markdown whether _ has any meaning is highly context sensitive and very difficult to get right with regular expressions alone.

And actual preprocessor in the form of a callback/hook that shipped the text out to some other command and read the results back would fix this. That would also enable much more advanced use cases.

[alerque]

By the way, not adding Markdown specific parsing makes some sense (I hadn't even thought of that when I wrote the issue and probably many other will assume Markdown as well especially considering the default changelog template is full on Markdown) but having a hook so that a full on preprocessor could be used would open the door to many more use cases.

[orhun]

@alerque do you suggest having something like this?

commit_preprocessors = [
    { pattern = '...', command = "pandoc -t commonmark" },
]

So that given command will be run and the result will be assigned to the commits that are matched with pattern.

If you can give some examples and further clarify your requirements I will be more than happy to implement it. Also, moving this to a separate issue would be nice 🐻