Shortcodes are evaluated instead of printed as text

[TobiasBg]

When adding a Shortcode to a core/code block, the Shortcode is printed as text instead of being evaluated (this is useful e.g. for documentation pages of a plugin's Shortcodes).

When then activating "Syntax-highlighting Code Block (with Server-side Rendering)" version 1.3.1 (tested on WP 6.1.1 and trunk), the Shortcode is however evaluated.

To reproduce this:

1. Add a small plugin that adds a Shortcode:

<?php
/*
Plugin Name: Shortcode Test
Description: Shortcode Test
Version: 1.0
*/

add_shortcode( 'test', 'shortcode_test' );
function shortcode_test( $atts, $content ) {
    return 'Just a test';
}

2. Add a "Code" block to a post or page with the Shortcode [test] as the content.
3. Compare the page output with "Syntax-highlighting Code Block (with Server-side Rendering)" deactivated and activated.

[TobiasBg]

My feeling is that the output that is generated by the plugin is then again run through do_shortcode() by WordPress, which does not seem to happen for the normal core/code block.

I then tried to use the escaped Shortcode [[test]], which however seems to have a core or Gutenberg bug: https://core.trac.wordpress.org/ticket/57351

As a temporary remedy, inserting a simple replacement

$content = str_replace( '[', '[', $content );

before

if ( ! DEVELOPMENT_MODE ) {
	set_transient( $transient_key, compact( 'content', 'attributes' ), MONTH_IN_SECONDS );
}

in the plugin's render_block() function seems to work.

[westonruter]

Thanks for the report. I'm not sure why this is happening, as we're not applying the_content filters or manually doing shortcodes with the output. I'll try to reproduce the issue in the new year.

You've also verified the issue happens when your shortcode plugin and this plugin are only active with a generic core theme? Is there any other plugin/theme that is causing the issue?

[TobiasBg]

Yes, all this is just with WP 6.1.1, the test Shortcode, and a default block theme.

My feeling (see the details in my previous reply) is that the output is run through the default the_content filters (which includes do_shortcode()), but as the [ HTML entity has been replaced by a [, that then gets recognized as a Shortcode.

[westonruter]

Ah, a Block theme. Can you try with an older theme, like Twenty Twenty One?

[TobiasBg]

Sure, just tested Twenty Twenty One, Twenty Twenty, Twenty Seventeen, and Twenty Ten.
Same problem everywhere, so it's not related to block themes. :-/

[westonruter]

Sorry for the delay. I can reproduce the issue.

This was also reported as an issue for Gutenberg in WordPress/gutenberg#13927 and fixed in WordPress/gutenberg#13996. The fix was for the block's save function to output [ instead of [. However, this is getting undone by us since we need the entities removed for passing into highlight.php:

syntax-highlighting-code-block/syntax-highlighting-code-block.php

Line 598 in b42cac4

$content = html_entity_decode( $matches['content'], ENT_QUOTES );

syntax-highlighting-code-block/syntax-highlighting-code-block.php

Lines 609 to 613 in b42cac4

	if ( $language ) {
	$r = $highlighter->highlight( $language, $content );
	} else {
	$r = $highlighter->highlightAuto( $content );
	}

So as you suggest, re-encoding the special characters may be the solution. Nevertheless, it'll have to be careful to not accidentally encode any of the markup that highlight.php is generating. This may require doing two passes, first to replace all tags with placeholders, then do the entity encoding, and then replace the placeholders with the original tags. I'll give it a shot.

[TobiasBg]

Thanks for looking into this! I've been using that str_replace() from #668 (comment) for now, and that seems to have helped a little bit (e.g. on this page) but not if that page content is loaded in a Loop (see this page and open the "The Shortcode [table id=N /]" section). But that might also be caused by Core#57351 or rather Core#55996 .

But something more robust than that str_replace() will probably be good.

[westonruter]

I've got a PR open to address the shortcode problem in #696

[westonruter]

This has been released on WordPress.org as part of v1.4.0.

[TobiasBg]

Great! Thanks, @westonruter!