Remove support for seeing non-ASCII whitespace as such

[maximbaz]

Subject of the issue

preserve leading and trailing no-break whitespaces in inline code: `   hello   `

Markdown strips regular whitespaces: hello
However it preserves non-breaking whitespaces:    hello   

Parser should also preserve non-breaking whitespaces, while stripping regular ones.

Steps to reproduce

astexplorer

Expected behaviour

{
  "type": "root",
  "children": [
    {
      "type": "paragraph",
      "children": [
        {
          "type": "text",
          "value": "preserve leading and trailing whitespaces in inline code: ",
        },
        {
          "type": "inlineCode",
          "value": "   hello   ",
        }
      ],
    }
  ],
}

Actual behaviour

{
  "type": "root",
  "children": [
    {
      "type": "paragraph",
      "children": [
        {
          "type": "text",
          "value": "preserve leading and trailing whitespaces in inline code: ",
        },
        {
          "type": "inlineCode",
          "value": "hello",
        }
      ],
    }
  ],
}

[wooorm]

Hey @maximbaz!

I tried it on the commonmark dingus (permalinking doesn’t work for me there) and it didn’t have your result. Why do you believe markdown should act like this?

P.S. GitHub does show them: preserve leading and trailing no-break whitespaces in inline code:    hello   

[maximbaz]

Hey, this all started because I have a variable with trailing whitespace as a default value, and this is documented in a markdown file (example below). I noticed that Github removes the trailing whitespace, and then I was suggested to use a non-breaking whitespace because Github keeps it. However, prettier (which uses this project internally) auto-formats my markdown document and removes the trailing whitespace, which is annoying 🙂

Now I see that apparently multiple markdown specifications exist (of course 🤦‍♂️ ), and not all of them preserve non-breaking whitespaces... What do you usually do in such cases? Follow commonmark's specification and not github's?

[maximbaz]

Actually, can it be a bug in commonmark dingus?

http://spec.commonmark.org/0.28/#code-spans

The contents of the code span are the characters between the two backtick strings, with leading and trailing spaces and line endings removed

http://spec.commonmark.org/0.28/#space

A space is U+0020.

[wooorm]

It could be! I dunno?!

Anyway, unrelated to this bug, but related to your original question: what I personally do in these cases of invisible characters is use the middot characters: foo·· (where · represents a space), similar to » for tab, ␊ for \n and ␍␊ for \r\n.

[maximbaz]

This is an interesting approach, thanks for sharing!

[wooorm]

Could you check with commonmark to see if it’s a bug in their code / dingus?

• If that’s a case, and they should follow GFM instead, I feel we should support this here too
• If it’s in fact GFM not following the commonmark spec, I’m not sure we should do something here...

[maximbaz]

It was easy, here's owner of commonmark repo explaining that non-breaking space should not be stripped, and this ticket in commonmark.js is describing what looks like a pretty much the same issue.