1.9.1: Gutenberg breaks "classic" posts w/ shortcodes by carelessly wrapping shortcodes into tags #3900

lkraav · 2017-12-10T17:21:34Z

(EDIT 2018-02-23 still broken in 2.2.0)
(EDIT 2018-03-06 still broken in 2.3.0)
(EDIT 2018-07-04 still broken in 3.1.1)
(EDIT 2018-07-21 still broken in 3.3.0)

Issue Overview

It is not safe to open classic posts in Gutenberg, shortcodes get unexpectedly wrapped in  tags.

Steps to Reproduce (for bugs)

Author a new post in classic editor w/ this example content

<h2>Shinier than yours, meatbag.</h2>

You can crush me but you can't crush my spirit! Is today's hectic lifestyle making you tense and impatient? It's a T. It goes "tuh". I'll tell them you went down prying the wedding ring off his cold, dead finger.

[column]

I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don't like lilacs! Your 'first' wife was the one who liked lilacs! No argument here. A true inspiration for the children.

[/column]

Hello world.

Open this post in Gutenberg
Switch to Code Editor mode to review transformed markup

Expected Behavior

I'm not sure what the latest transformation goal for classic posts is.  seems to be gone? Is #2454 relevant to the issue at hand?

Current Behavior

I see the following transformation in Code Editor, which I consider broken for the shortcodes, since the result is broken rendering on the frontend:

<h2>Shinier than yours, meatbag.</h2>
<p>You can crush me but you can't crush my spirit! Is today's hectic lifestyle making you tense and impatient? It's a T. It goes "tuh". I'll tell them you went down prying the wedding ring off his cold, dead finger.</p>
<p>[column grid="2" span="1"]</p>
<p>I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don't like lilacs! Your 'first' wife was the one who liked lilacs! No argument here. A true inspiration for the children.</p>
<p>[/column]</p>
<p>[column grid="2" span="1"]</p>
<p>Hello world.</p>
<p>[/column]</p>

"Convert to blocks" also gets completely confused by this result and fails to separate shortcodes into their own blocks.

Possible Solution

Not sure, could use pointers. What instructions should I provide users to avoid insta-breaking all classic content saved in Gutenberg?

Screenshots / Video

N/A

Related Issues and/or PRs

Not sure, could use pointers.

The text was updated successfully, but these errors were encountered:

lkraav · 2017-12-11T19:23:32Z

Tested on 1.9.0, situation is exactly the same.

aduth · 2017-12-18T20:58:51Z

Does the classic editor handle this gracefully when viewing a post with this markup on the front-end?

lkraav · 2017-12-19T14:22:41Z

Does the classic editor handle this gracefully when viewing a post with this markup on the front-end?

Yes. (Maybe also have to account for https://core.trac.wordpress.org/ticket/14050)

@aduth but may I ask, how is this question related to Gutenberg wrapping shortcodes into  in the backend editor. Isn't frontend rendering a different topic?

aduth · 2017-12-19T14:55:31Z

@lkraav My question was to try to clarify whether the Gutenberg behavior is in-fact the expected behavior, depending on how this would have been treated for pre-Gutenberg content.

lkraav · 2017-12-19T15:01:43Z

My question was to try to clarify whether the Gutenberg behavior is in-fact the expected behavior, depending on how this would have been treated for pre-Gutenberg content.

@aduth OK, but wouldn't the backend-relevant question be about "does classic editor have the same effect when switching between Text vs Visual modes"?

But indeed, if you're saying Gutenberg backend transforms are already supposed to accurately mirror frontend rendering output, then I do see the point of the question, but how can this broken paragraph wrapping be expected output? Such expected output would have been breaking HTML for all opening+closing block level shortcodes for over a decade, no?

azaozz · 2018-01-02T15:57:22Z

Does the classic editor handle this gracefully when viewing a post with this markup on the front-end?

The answer is "technically: no" because this is handled by shortcode_unautop() on display, not by the editor. Shortcodes and their content that contain line breaks (doing it wrong?) are still wrapped in  tags inside the editor and "unwrapped" before displaying the posts.

Seems Gutenberg would need some sort of shortcode_unautop() when loading existing posts. Not so sure what to do with shortcodes without "preview" after loading though. Perhaps a better idea is to "force-convert" all shortcodes to blocks. That will protect badly written shortcodes and show "previews" when supported, all without the need of more code/processing.

lkraav · 2018-02-23T16:11:59Z

Tested broken in 2.2.0.

(EDIT 2018-03-06 Tested broken in 2.3.0)

danielbachhuber · 2018-06-18T13:34:33Z

@lkraav I took a look into this issue this morning.

Based on my understanding, the problem of Gutenberg breaks "classic" posts w/ shortcodes by carelessly wrapping shortcodes into tags would be fixed by support for multi-line shortcodes (#4942). When Gutenberg supports multi-line shortcodes, then your example will transform into:

<!-- wp:heading -->
<h2>Shinier than yours, meatbag.</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>You can crush me but you can't crush my spirit! Is today's hectic lifestyle making you tense and impatient? It's a T. It goes "tuh". I'll tell them you went down prying the wedding ring off his cold, dead finger.</p>
<!-- /wp:paragraph -->

<!-- wp:shortcode -->
[column]

I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don't like lilacs! Your 'first' wife was the one who liked lilacs! No argument here. A true inspiration for the children.

[/column]
<!-- /wp:shortcode -->

<!-- wp:paragraph -->
<p>Hello world.</p>
<!-- /wp:paragraph -->

Is my statement an accurate representation of your expectation, or is there additional detail I'm missing?

lkraav · 2018-06-19T10:06:01Z

<!-- wp:shortcode -->
[column]

I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don't like lilacs! Your 'first' wife was the one who liked lilacs! No argument here. A true inspiration for the children.

[/column]
<!-- /wp:shortcode -->

Thanks for taking it on. Above output looks a bit suspect to me, because how would that look like in Visual Mode? We def. need to keep block-style visual editing intact between the opening and closing shortcodes (=== all blocks visual usage available). I think we'd be heading into Shortcake / Nested blocks territory here, which is a whole world on its own.

Expected output for me, at least for starters, would be:

<!-- wp:shortcode -->
[column]
<!-- /wp:shortcode -->

<!-- wp:paragraph -->
<p>I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don't like lilacs! Your 'first' wife was the one who liked lilacs! No argument here. A true inspiration for the children.</p>
<!-- /wp:paragraph -->

<!-- wp:shortcode -->
[/column]
<!-- /wp:shortcode -->

swissspidy · 2018-06-19T10:17:39Z

Possibly related: #5940.

danielbachhuber · 2018-06-19T16:59:59Z

Above output looks a bit suspect to me, because how would that look like in Visual Mode? We def. need to keep block-style visual editing intact between the opening and closing shortcodes (=== all blocks visual usage available).

Oh, I think we have different expectations here.

I don't expect you'll be able to visually edit the inner content of a shortcode. Your UI will look more like this:

Specifically, the shortcode block adopts a <textarea> instead of <input>.

Visually editing the inner content of a shortcode sounds like Pandora's box to me.

lkraav · 2018-06-19T21:34:56Z

Visually editing the inner content of a shortcode sounds like Pandora's box to me.

Perhaps, but such a UX would be a world ahead of being restricted to plain text and/or html between opening and closing shortcodes. Isn't this exactly what Gutenberg is trying to get away from?

OTOH, if shortcodes are being phased out for blocks, text-only could be a decent enough of a stop-gap solution.

Primary goal should be to just keep markup from radically breaking, such as shown in the opening comment.

(EDIT 2018-07-04 Tested broken in 3.1.1)

danielbachhuber · 2018-06-19T21:35:47Z

OTOH, if shortcodes are being phased out for blocks, text-only could be a decent enough of a stop-gap solution.

Correct, this is my thinking.

Primary goal should be to just keep markup from radically breaking, such as shown in the opening comment.

Yes, I agree.

danielbachhuber · 2018-07-17T13:41:09Z

With #7892, this Classic text:

[column]

I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don't like lilacs! Your 'first' wife was the one who liked lilacs! No argument here. A true inspiration for the children.

[/column]

Transforms into the following block:

<!-- wp:shortcode -->
[column]</p>
<p>I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don't like lilacs! Your 'first' wife was the one who liked lilacs! No argument here. A true inspiration for the children.</p>
<p>[/column]
<!-- /wp:shortcode -->

Progress, but still not quite there.

lkraav · 2018-07-17T14:19:18Z

Progress, but still not quite there.

Thanks a ton for pushing this.

Reminds me of the work I did for https://core.trac.wordpress.org/ticket/14050 which has stalled for years, because the risk surface and complexity of regexing through shortcodes is massive.

I still use my own micro-plugin I posted there on all sites, as my mini-updates to the monster regex have proven to be completely production safe, and eliminated most of those  closing tags appearing out of nowhere.

But that's PHP shortcode_unautop() and you're working this I think at fully JS level. Still, *shivers* :)

danielbachhuber · 2018-07-18T13:58:52Z

Some conversation with @azaozz in the thread of https://wordpress.slack.com/messages/C02QB2JS7/

The suggestion to explore is:

azaozz 24 hours ago hm, ok, why don't we do the wpautop() on the server when loading old posts, then run the post_content through shortcode_unautop()
azaozz 24 hours ago this is exactly the same as now on the front-end

pento · 2018-07-20T05:09:14Z

If a fix is bad, but it works, is it actually bad? 🙂

$ git diff
diff --git a/core-blocks/shortcode/index.js b/core-blocks/shortcode/index.js
index 3f8d0b049..829f34d48 100644
--- a/core-blocks/shortcode/index.js
+++ b/core-blocks/shortcode/index.js
@@ -46,7 +46,7 @@ export const settings = {
                                        text: {
                                                type: 'string',
                                                shortcode: ( attrs, { content } ) => {
-                                                       return content;
+                                                       return wp.oldEditor.removep( wp.oldEditor.autop( content ) );
                                                },
                                        },
                                },

lkraav · 2018-07-20T10:12:51Z

If a fix is bad, but it works, is it actually bad?

Lol, sounds similar to "violence is not the answer... but it works!"

I still don't know the general answer to either.

danielbachhuber · 2018-07-20T11:51:40Z

If a fix is bad, but it works, is it actually bad?

🎉 Honestly, I think it's better than @azaozz because it limits the scope of the change. With #3900 (comment), I'd be slightly terrified of something breaking in an unexpected way.

pento · 2018-07-20T11:57:05Z

I've done basically no testing on this fix, just a few cases to show that it removes all the 's that get added to shortcodes containing plain text.

Are there more complex cases that we need to deal with? Nested shortcodes, shortcodes with HTML inside them, that kind of thing?

danielbachhuber · 2018-07-20T12:06:43Z

I've done basically no testing on this fix, just a few cases to show that it removes all the
's that get added to shortcodes containing plain text.

I think it's a reasonable assumption that  tags are excluded/stripped from shortcode inner_content, given this is the current behavior of the Classic Editor:

Are there more complex cases that we need to deal with? Nested shortcodes, shortcodes with HTML inside them, that kind of thing?

"Complex cases" is shortcode's middle name. I think #8077 is better than what we have now, and we'll discover whether there are other edge cases to accommodate for when Gutenberg sees a wider audience.

aduth · 2018-07-20T13:48:21Z

Transforms into the following block:

<!-- wp:shortcode -->
[column]</p>
<p>I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I >don't like lilacs! Your 'first' wife was the one who liked lilacs! No argument here. A true inspiration for the children.</p>
<p>[/column]
<!-- /wp:shortcode -->

While it surely looks strange, I don't know that this is strictly wrong.

With Gutenberg completely disabled, this is exactly what's passed as the $content argument to the callback of add_shortcode provided the original (non-block) markup:

<?php

/**
 * Plugin Name: Demo Shortcode
 */

add_shortcode( 'column', function( $attrs, $content ) {
	error_log( $content );
	return 'foo';
} );

[20-Jul-2018 13:43:02 UTC] </p>
<p>I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don&#8217;t like lilacs! Your &#8216;first&#8217; wife was the one who liked lilacs! No argument here. A true inspiration for the children.</p>
<p>

So it doesn't seem unexpected that Gutenberg, with its ignoring wpautop, would render the p tags verbatim. The content passed to the add_shortcode callback is the same.

[20-Jul-2018 13:45:46 UTC] </p>
<p>I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don&#8217;t like lilacs! Your &#8216;first&#8217; wife was the one who liked lilacs! No argument here. A true inspiration for the children.</p>
<p>

In fact, it's only with the proposed removep behavior in #8077 that the content actually changes, which to me seems like the real regression:

[20-Jul-2018 13:45:53 UTC] 

I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don&#8217;t like lilacs! Your &#8216;first&#8217; wife was the one who liked lilacs! No argument here. A true inspiration for the children.

This may be partly due to how wpautop is disabled for posts containing blocks.

gutenberg/lib/compat.php

Lines 134 to 149 in 5a65727

    
           /** 
        
            * As a substitute for the default content `wpautop` filter, applies autop 
        
            * behavior only for posts where content does not contain blocks. 
        
            * 
        
            * @param  string $content Post content. 
        
            * @return string          Paragraph-converted text if non-block content. 
        
            */ 
        
           function gutenberg_wpautop( $content ) { 
        
           	if ( gutenberg_content_has_blocks( $content ) ) { 
        
           		return $content; 
        
           	} 
        
           	return wpautop( $content ); 
        
           } 
        
           remove_filter( 'the_content', 'wpautop' ); 
        
           add_filter( 'the_content', 'gutenberg_wpautop', 8 );

danielbachhuber · 2018-07-20T13:58:24Z

With Gutenberg completely disabled, this is exactly what's passed as the $content argument to the callback of add_shortcode provided the original (non-block) markup:

<?php

/**
 * Plugin Name: Demo Shortcode
 */

add_shortcode( 'column', function( $attrs, $content ) {
	error_log( $content );
	return 'foo';
} );

[20-Jul-2018 13:43:02 UTC] </p>
<p>I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don&#8217;t like lilacs! Your &#8216;first&#8217; wife was the one who liked lilacs! No argument here. A true inspiration for the children.</p>
<p>

This assessment is correct.

@lkraav Can you share more detail about how you handle this in the Classic Editor, and what your expectations would be about Gutenberg?

@aduth Do you have a proposed resolution to suggest at this point? maybelater?

aduth · 2018-07-20T14:14:00Z

It's tricky, since the shortcode rendering itself is completely unimpacted, but also would have anticipated that the rest of content would have also had wpautop applied. So while previously, the developer could return $content directly and have a consistent tags, if they were to do so now, without the wpautop applied by Gutenberg, it could result in some breakage:

I was inclined to think that this would break, but in fact the rendered output is identical (though both the "original" and "converted" rendered outputs have non-balanced tags). I think this is because the server-side shortcode renderer replaces the opening  anyways (citation needed). At this point, I'd suggest that this is "working as intended", at least so far as backwards-compatibility.

Original:

<h2>Shinier than yours, meatbag.</h2>

You can crush me but you can't crush my spirit! Is today's hectic lifestyle making you tense and impatient? It's a T. It goes "tuh". I'll tell them you went down prying the wedding ring off his cold, dead finger.

[column]

I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don't like lilacs! Your 'first' wife was the one who liked lilacs! No argument here. A true inspiration for the children.

[/column]

Hello world.

Converted:

<!-- wp:heading -->
<h2>Shinier than yours, meatbag.</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>You can crush me but you can't crush my spirit! Is today's hectic lifestyle making you tense and impatient? It's a T. It goes "tuh". I'll tell them you went down prying the wedding ring off his cold, dead finger.</p>
<!-- /wp:paragraph -->

<!-- wp:shortcode -->
[column]</p>
<p>I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don't like lilacs! Your 'first' wife was the one who liked lilacs! No argument here. A true inspiration for the children.</p>
<p>
[/column]
<!-- /wp:shortcode -->

<!-- wp:paragraph -->
<p>Hello world.</p>
<!-- /wp:paragraph -->

Original Rendered:

<h2>Shinier than yours, meatbag.</h2>
<p>You can crush me but you can&#8217;t crush my spirit! Is today&#8217;s hectic lifestyle making you tense and impatient? It&#8217;s a T. It goes &#8220;tuh&#8221;. I&#8217;ll tell them you went down prying the wedding ring off his cold, dead finger.</p>
</p>
<p>I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don&#8217;t like lilacs! Your &#8216;first&#8217; wife was the one who liked lilacs! No argument here. A true inspiration for the children.</p>
<p>
<p>Hello world.</p>

Converted Rendered:

<h2>Shinier than yours, meatbag.</h2>

<p>You can crush me but you can&#8217;t crush my spirit! Is today&#8217;s hectic lifestyle making you tense and impatient? It&#8217;s a T. It goes &#8220;tuh&#8221;. I&#8217;ll tell them you went down prying the wedding ring off his cold, dead finger.</p>

</p>
<p>I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don&#8217;t like lilacs! Your &#8216;first&#8217; wife was the one who liked lilacs! No argument here. A true inspiration for the children.</p>
<p>


<p>Hello world.</p>

danielbachhuber · 2018-07-20T14:25:46Z

Ok. My one remaining concern at this point is that a user who sees  paragraph tags exposed in the transformed Shortcode Block may think them unexpected, remove them, and then experience different behavior in rendering.

Removing the  paragraph tags in that example changes the inner content from broken / unbalanced tags to no tags at all:

[20-Jul-2018 14:19:33 UTC] '</p>
<p>I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don&#8217;t like lilacs! Your &#8216;first&#8217; wife was the one who liked lilacs! No argument here. A true inspiration for the children.</p>
<p>'
[20-Jul-2018 14:19:45 UTC] '
I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don&#8217;t like lilacs! Your &#8216;first&#8217; wife was the one who liked lilacs! No argument here. A true inspiration for the children.
'

Generally, I think the expected behavior is:

No  paragraph tags are displayed in shortcode inner content in the editor.
(Unbalanced / broken)  paragraph tags are included in the $content passed to the shortcode render callback.

At this point though, I'd be fine to close as maybelater for now and see if becomes a more widespread problem. We have the beginnings of a fix at least, so we're at a better state than we were a day ago.

Or, we could land #8077 and also recreate unbalanced / broken paragraph tags on the inner content prior to rendering the shortcode.

I'll wait for @lkraav to weigh in before closing.

lkraav · 2018-07-21T15:42:17Z

I rebased 3900-unautop on top of today's master, which is basically v3.3.0 + 201be4b. So #8077 is included.

First thing noticed: opening comment use case still doesn't work off the bat. If the user just opens the example document in Gutenberg, saves it, markup will become silently broken, esp. if no editing operation was performed.

Looking at #3900 (comment) it seems like it somehow became the expectation that the user should know to execute "Convert to Blocks" operation. No such assumption can be safely made, but I'm betting this is known so perhaps I'm misreading where exactly we're at.

First and foremost, I would still expect Gutenberg to be able to maintain original comment document health.

Convert to blocks

Good news is, #8077 seems to indeed now provide correct transformation on the opening comment example document. Nice step forward, for sure. This working well would help admins all across the world perform safer one-click getaways from legacy markup. Can't imagine shipping 5.0 with anything less.

Stretch goal convert

Sure would be dreamy if shortcode content would run through transformation chain as well. Feels like the infrastructure is already in place, anything specific stopping it?

Dreamy output:

<!-- wp:shortcode -->
[column grid="2" span="1"]
<!-- /wp:shortcode -->

<!-- wp:paragraph -->
<p>I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don't like lilacs! Your 'first' wife was the one who liked lilacs! No argument here. A true inspiration for the children.</p>
<!-- /wp:paragraph -->

<!-- wp:shortcode -->
[/column]
<!-- /wp:shortcode -->

<!-- wp:shortcode -->
[column grid="2" span="1"]
<!-- /wp:shortcode -->

<!-- wp:paragraph -->
<p>Hello world.</p>
<!-- /wp:paragraph -->

<!-- wp:shortcode -->
[/column]
<!-- /wp:shortcode -->

aduth · 2018-07-26T00:57:01Z

If the user just opens the example document in Gutenberg, saves it, markup will become silently broken, esp. if no editing operation was performed.

Can we clarify what is specifically broken? The markup changes, introducing paragraph tags which did not exist previously, but in what way is this not effectively the same as it was previously? When taking the markup from the original comment and viewing the output both as input into the Classic editor and then saved again in Gutenberg, I see no difference.

lkraav · 2018-07-26T09:05:44Z

When taking the markup from the original comment and viewing the output both as input into the Classic editor and then saved again in Gutenberg, I see no difference.

@aduth

do you have Classic Editor plugin enabled? I've seen this make a significant behavior difference in the past. Either way, all my testing is without Classic Editor plugin (btw this plugin is quite confusing to talk about in the context of classic editor), because I've not seen anywhere that it's going to be required to have a non-markup-breaking Gutenberg installation.
which Gutenberg branch, version, commit did you run your observation on?

Here's the breakage, using 3900-unautop branch:

ORIGINAL CONTENT

<h2>Shinier than yours, meatbag.</h2>

You can crush me but you can't crush my spirit! Is today's hectic lifestyle making you tense and impatient? It's a T. It goes "tuh". I'll tell them you went down prying the wedding ring off his cold, dead finger.

[column]

I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don't like lilacs! Your 'first' wife was the one who liked lilacs! No argument here. A true inspiration for the children.

[/column]

Hello world.

GUTENBERG WRAPS SHORTCODES IN 

<h2>Shinier than yours, meatbag.</h2>
<p>This is a post <strong>WITHOUT</strong> Gutenberg markup.</p>
<p><a href="https://github.com/WordPress/gutenberg/issues/3900">#3900</a></p>
<p>You can crush me but you can't crush my spirit! Is today's hectic lifestyle making you tense and impatient? It's a T. It goes "tuh". I'll tell them you went down prying the wedding ring off his cold, dead finger.</p>
<p>[column grid="2" span="1"]</p>
<p>I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don't like lilacs! Your 'first' wife was the one who liked lilacs! No argument here. A true inspiration for the children.</p>
<p>[/column]</p>
<p>[column grid="2" span="1"]</p>
<p>Hello world.</p>
<p>[/column]</p>

I do see @pento pushing a new (to me) strategy in #8077 (comment) maybe this a game-changer here?

pento · 2018-07-26T11:18:21Z

@lkraav: Yah, we did a bit of brainstorming earlier, and came up with this solution. If you could test out the current state of #8077 and let us know if it behaves correctly for you, that'd be super helpful. 🙂

lkraav · 2018-07-26T11:33:17Z

Yah, we did a bit of brainstorming earlier, and came up with this solution. If you could test out the current state of #8077 and let us know if it behaves correctly for you, that'd be super helpful. slightly_smiling_face

Thanks, can attend in 2 hrs.

lkraav · 2018-07-26T14:03:03Z

If you could test out the current state of #8077 and let us know if it behaves correctly for you, that'd be super helpful.

@pento

it seems like there's improvement in the editor

switching to Code Editor avoids changing markup almost 100% - line break after <h2> still gets removed
... but this change surprisingly does not get saved when you click Update. It's like 99% WYSIWYG.

frontend output is still rendering the shortcode with broken  tags

		<h2>Shinier than yours, meatbag.</h2>
<p>This is a post <strong>WITHOUT</strong> Gutenberg markup.</p>
<p><a href="https://github.com/WordPress/gutenberg/issues/3900">#3900</a></p>
<p>You can crush me but you can&#8217;t crush my spirit! Is today&#8217;s hectic lifestyle making you tense and impatient? It&#8217;s a T. It goes &#8220;tuh&#8221;. I&#8217;ll tell them you went down prying the wedding ring off his cold, dead finger.</p>
<div class="column-grid column-grid-2"><div class="column column-span-1 column-push-0 column-first"></p><!-- HERE -->
<p>I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don&#8217;t like lilacs! Your &#8216;first&#8217; wife was the one who liked lilacs! No argument here. A true inspiration for the children.</p>
<p>
</div>
<div class="column column-span-1 column-push-0 column-last"></p><!-- HERE -->
<p>Hello world.</p>
<p>
</div></div>

Fortunately, this break seems to be https://core.trac.wordpress.org/ticket/14050 territory. After activating my "improved regex" microplugin, broken paragraphs disappear as they should (and have for the past x years).

I also verified w/ 3.3.0 - breakage is the same, so #8077 or 3900-autop branch does not have a regression there.

SUMMARY

Considering the only thing missing is the "stretch goal" of "Convert to blocks" also being able to blockify content between enclosing shortcodes

[shortcode]
I want be blockified, too, but not there yet.
[/shortcode]

I'd suggest calling #3900 resolved and I file a new issue for enclosing shortcode blockification.

It's definitely a big win that we got "Convert to blocks" simply convert shortcodes better right now, even if the content in there stays as is. Frontend rendering the_content filters fortunately should help it out.

danielbachhuber · 2018-07-26T14:41:39Z

Sounds good, thanks @lkraav

When transforming classic content to a shortcode block, or having a post with a single freeform block in it, `post_content` ended up in a state that wasn't backward compatible. This change ensures that un-blockified posts can continue to be used in the Classic Editor as expected, and that the shortcode block both renders in Gutenberg correctly, and on the front end. Fixes #3900.

lkraav mentioned this issue Dec 10, 2017

1.9.1: Classic "Visual" editor litters Gutenberg markup w/ superfluous `´ tags #3901

Closed

lkraav changed the title ~~1.8.1: Gutenberg breaks "classic" posts w/ shortcodes by carelessly wrapping shortcodes into tags~~ 1.9.0: Gutenberg breaks "classic" posts w/ shortcodes by carelessly wrapping shortcodes into tags Dec 11, 2017

lkraav changed the title ~~1.9.0: Gutenberg breaks "classic" posts w/ shortcodes by carelessly wrapping shortcodes into tags~~ 1.9.1: Gutenberg breaks "classic" posts w/ shortcodes by carelessly wrapping shortcodes into tags Dec 14, 2017

bobbingwide mentioned this issue Jan 14, 2018

New lines in shortcode content are ignored #4456

Closed

2 tasks

jeffpaul added the [Type] Bug An existing feature does not function as intended label Jan 26, 2018

danielbachhuber added the [Feature] Shortcodes Related to shortcode functionality label Apr 10, 2018

danielbachhuber added this to the Merge Proposal milestone Apr 10, 2018

danielbachhuber added the [Type] Try Gutenberg Blocker label Apr 11, 2018

danielbachhuber self-assigned this May 31, 2018

danielbachhuber mentioned this issue Jun 5, 2018

Blockers for launching "Try Gutenberg" callout in core #7147

Closed

danielbachhuber mentioned this issue Jun 18, 2018

[2.9.2] Convert to Blocks not properly converting successive Shortcodes #7030

Closed

danielbachhuber mentioned this issue Jul 11, 2018

On installation in an existing website, make intelligent choice whether a post or page contains classic or raw code #7859

Closed

mtias modified the milestones: Merge Proposal: Editor, Try Callout Jul 19, 2018

mtias removed the [Type] Try Gutenberg Blocker label Jul 19, 2018

danielbachhuber mentioned this issue Jul 20, 2018

Run wp.oldEditor.removep() when transforming shortcodes #8077

Merged

4 tasks

pento closed this as completed in #8077 Jul 27, 2018

lkraav mentioned this issue Jul 31, 2018

Add Block: Shortcode #1709

Closed

gwwar mentioned this issue Aug 24, 2018

Gutenberg: Edit an existing post made by the Calypso Editor without mangling content Automattic/wp-calypso#26909

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

1.9.1: Gutenberg breaks "classic" posts w/ shortcodes by carelessly wrapping shortcodes into <p> tags #3900

1.9.1: Gutenberg breaks "classic" posts w/ shortcodes by carelessly wrapping shortcodes into <p> tags #3900

lkraav commented Dec 10, 2017 •

edited

Loading

lkraav commented Dec 11, 2017

aduth commented Dec 18, 2017

lkraav commented Dec 19, 2017

aduth commented Dec 19, 2017

lkraav commented Dec 19, 2017 •

edited

Loading

azaozz commented Jan 2, 2018

lkraav commented Feb 23, 2018 •

edited

Loading

danielbachhuber commented Jun 18, 2018

lkraav commented Jun 19, 2018 •

edited

Loading

swissspidy commented Jun 19, 2018

danielbachhuber commented Jun 19, 2018

lkraav commented Jun 19, 2018 •

edited

Loading

danielbachhuber commented Jun 19, 2018

danielbachhuber commented Jul 17, 2018 •

edited

Loading

lkraav commented Jul 17, 2018 •

edited

Loading

danielbachhuber commented Jul 18, 2018

pento commented Jul 20, 2018

lkraav commented Jul 20, 2018 •

edited

Loading

danielbachhuber commented Jul 20, 2018

pento commented Jul 20, 2018

danielbachhuber commented Jul 20, 2018

aduth commented Jul 20, 2018 •

edited

Loading

danielbachhuber commented Jul 20, 2018 •

edited

Loading

aduth commented Jul 20, 2018

danielbachhuber commented Jul 20, 2018

lkraav commented Jul 21, 2018

aduth commented Jul 26, 2018

lkraav commented Jul 26, 2018 •

edited

Loading

pento commented Jul 26, 2018

lkraav commented Jul 26, 2018

lkraav commented Jul 26, 2018 •

edited

Loading

danielbachhuber commented Jul 26, 2018

1.9.1: Gutenberg breaks "classic" posts w/ shortcodes by carelessly wrapping shortcodes into <p> tags #3900

1.9.1: Gutenberg breaks "classic" posts w/ shortcodes by carelessly wrapping shortcodes into <p> tags #3900

Comments

lkraav commented Dec 10, 2017 • edited Loading

Issue Overview

Steps to Reproduce (for bugs)

Expected Behavior

Current Behavior

Possible Solution

Screenshots / Video

Related Issues and/or PRs

lkraav commented Dec 11, 2017

aduth commented Dec 18, 2017

lkraav commented Dec 19, 2017

aduth commented Dec 19, 2017

lkraav commented Dec 19, 2017 • edited Loading

azaozz commented Jan 2, 2018

lkraav commented Feb 23, 2018 • edited Loading

danielbachhuber commented Jun 18, 2018

lkraav commented Jun 19, 2018 • edited Loading

swissspidy commented Jun 19, 2018

danielbachhuber commented Jun 19, 2018

lkraav commented Jun 19, 2018 • edited Loading

danielbachhuber commented Jun 19, 2018

danielbachhuber commented Jul 17, 2018 • edited Loading

lkraav commented Jul 17, 2018 • edited Loading

danielbachhuber commented Jul 18, 2018

pento commented Jul 20, 2018

lkraav commented Jul 20, 2018 • edited Loading

danielbachhuber commented Jul 20, 2018

pento commented Jul 20, 2018

danielbachhuber commented Jul 20, 2018

aduth commented Jul 20, 2018 • edited Loading

danielbachhuber commented Jul 20, 2018 • edited Loading

aduth commented Jul 20, 2018

danielbachhuber commented Jul 20, 2018

lkraav commented Jul 21, 2018

Convert to blocks

Stretch goal convert

aduth commented Jul 26, 2018

lkraav commented Jul 26, 2018 • edited Loading

pento commented Jul 26, 2018

lkraav commented Jul 26, 2018

lkraav commented Jul 26, 2018 • edited Loading

SUMMARY

danielbachhuber commented Jul 26, 2018

lkraav commented Dec 10, 2017 •

edited

Loading

lkraav commented Dec 19, 2017 •

edited

Loading

lkraav commented Feb 23, 2018 •

edited

Loading

lkraav commented Jun 19, 2018 •

edited

Loading

lkraav commented Jun 19, 2018 •

edited

Loading

danielbachhuber commented Jul 17, 2018 •

edited

Loading

lkraav commented Jul 17, 2018 •

edited

Loading

lkraav commented Jul 20, 2018 •

edited

Loading

aduth commented Jul 20, 2018 •

edited

Loading

danielbachhuber commented Jul 20, 2018 •

edited

Loading

lkraav commented Jul 26, 2018 •

edited

Loading

lkraav commented Jul 26, 2018 •

edited

Loading