Fix fill-region for code blocks

[vermiculus]

As we've discussed, fill-region doesn't respect the mode's definition of paragraphs. I'm opening this issue to discuss workaround or other steps forward as this has become quite a nuisance for Magithub (and other markdown-mode-based packages, I'm sure).

One possibility I stumbled upon while trying to implement my own fill-function was the use of 'hard' newlines:

use-hard-newlines is an interactive compiled Lisp function in
‘paragraphs.el’.

(use-hard-newlines &optional ARG INSERT)

Toggle distinguishing between hard and soft newlines.
With a prefix argument ARG, enable the feature if ARG is
positive, and disable it otherwise.  If called from Lisp, enable
it if ARG is omitted or nil.

When enabled, the functions ‘newline’ and ‘open-line’ add the
text-property ‘hard’ to newlines that they insert, and a line is
only considered as a candidate to match ‘paragraph-start’ or
‘paragraph-separate’ if it follows a hard newline.

When enabling, if there are newlines in the buffer but no hard
newlines, ask the user whether to mark as hard any newlines
preceding a ‘paragraph-start’ line.  From a program, second arg
INSERT specifies whether to do this; it can be ‘never’ to change
nothing, t or ‘always’ to force marking, ‘guess’ to try to do the
right thing with no questions, nil or anything else to ask the
user.

Newlines not marked hard are called "soft", and are always internal
to paragraphs.  The fill functions insert and delete only soft newlines.

It looks like if the newlines in codeblocks are 'hard', then they won't fill. Could we introduce some pre-processing to add that text property to newlines inside code-blocks? Perhaps then fill-region will work without any additional futzing.

[jrblevin]

Thanks, this is definitely interesting and I will take a look.

(For my own reference later, I'll just mention that this was related to #169.)

[jrblevin]

I tried setting the hard property on newlines in code blocks and setting use-hard-newlines to t. Unfortunately, then fill-region then ignores newlines elsewhere that don't have the hard property. In other words, it will combine multiple regular paragraphs together as if there was no paragraph break. So, it seems we can't have it both ways.

Another option might be to do some clever tricks in the markdown-fill-forward-paragraph function, but that logic is evading me right now.

[jrblevin]

I managed to hack the behavior of markdown-mode's fill-forward-paragraph-function to make this work. See the commit message if you're interested in the details.

[vermiculus]

Sorry for the delay, but sweet! I was just trying this out after reading #251 and its mention of font-lock-ensure. For posterity, I came up with the following function:

(defun my-gfm-fill (text)
  (with-temp-buffer
    (gfm-mode)
    (insert text)
    (font-lock-ensure)
    (fill-region (point-min) (point-max))
    (buffer-string)))

Seems to do the trick 👍