Enhance the shortcode block to support previewing of shortcodes #4710

niranjan-uma-shankar · 2018-01-27T22:51:52Z

Description

This PR adds the capability to preview shortcodes entered in the Shortcode block. This is similar to previews rendered by the embed block and custom HTML block. Work done in this PR references the issue #1054

Summary of changes

I've added a new REST API endpoint that converts the shortcode to its output filtered through its hooks
- GET /gutenberg/v1/shortcodes - get the filtered output for a given shortcode
I've changed the UI of the shortcode block to a tabbed component (identical to the custom HTML block). I've posted a comment on my thoughts behind this design consideration. The comment has screenshots of the UI, please check there.
The shortcode preview is embedded in a Sandbox component. This component (which is essentially creating an iframe) needed to support injection of custom JS and styles, since some shortcodes such as [gallery] and [playlist] need additional styles and scripts, such as the theme style or mediaelement JS. I've made the necessary changes to support this.

How to test

Test with all the default shortcodes - [audio], [video], [caption], [wp_caption], [embed], [gallery], [playlist type="audio"], [playlist type="video"].
Add various attributes to the shortcodes and check if they get applied. To note, shortcodes entered without any src attributes will try to automatically render an attached media file of the corresponding type. For eg. :
(a) [audio] will render the attached audio vs [audio src="audio-source.mp3"]
(b) [gallery] will render all attached images, whereas [gallery ids="10,11,12"] will render those specific image ids.
Create a custom shortcode via add_shortcode and use it in the shortcode block. Also, get a little fancy with Shortcode-ception.
Edit/modify/replace shortcodes in the block - clicking preview after every such action should render the most recent version of the shortcode
Non-existent shortcodes will be treated as a normal text and displayed
If there's no attached media/photo to a post, then using just [gallery] or [audio] (without "src" or "ids" attributes) will give an error message that the preview cannot be rendered
Giving incompatible "ids" - for example, in [gallery ids="11"], if 11 refers to an audio file, or in [playlist ids="12" type="video"], "12" refers to an audio file, then a message will be flashed "Sorry, couldn't render preview"

Screenshot

Below is an example of a preview for [caption]

Known issue

1. Preview for [wp-playlist] does not work. I'm trying to inject all the necessary JS into the Sandbox component (the js is fetched by the rest endpoint and can be seen in these lines of code). I get an error Uncaught TypeError: Cannot read property 'replace' of undefined in underscore.min.js. Any pointers here would be appreciated.

This commit is towards rendering a front end preview for shortcode blocks. The shortcode block now implements a tabbed preview option, similar to HTML blocks. The user can edit their shortcodes, and previewing again will re-render the edited shortcode. Works for embed shortcodes too. Known issues - (1) playlist shortcode doesn't work (2) the iframe height/width in the preview tab needs to wrap content size. For example, the iframe is too big when previewing an audio player using audio shortcode (3) gallery shortcode preview stacks the images vertically instead of horizontally (4) video shortcode doesn't work for URLs supported by oembed

…standards

…ortcode

Previously, the post ID was fetched by processing the post's URL and parsing the post ID GET parameter. Now, a better approach is taken by reading the post ID value from the redux store.

In this revision, custom css and js files are injected as props to the iframe sandbox. Custom css and js is needed in certain cases, for example, [gallery] shortcode needs the parent theme style.css.

Shortcode content type (if it's a video or otherwise), and the shortcode's custom css and js (if any) are fetched and returned as parameters to the front end. In the case of [gallery] and [caption], for example, the theme's style.css is needed, else the shortcode preview will not render properly. [playlist] needs mediaelement JS to be able to render the playlist components.

1) Certain shortcodes, such as [gallery], [caption] and [playlist] need styles and JS inside of the sandbox iframe to be able to render content appropriately. The sandbox component didn't support injection of custom links for external stylesheets and scripts, so this revision attempts to fix that 2) [audio] shortcode fails to render since bounding client width is 0 (weird, I know). So, in this revision, I add a null check and set a minimum width if bounding client width or height is zero.

niranjan-uma-shankar · 2018-01-28T15:37:34Z

For the [embed] shortcode - I'd like to see a message asking people to check if there is an embed block for what they're embedding.

Good suggestion! We can add a message that guides users to use the embed block.

The other points you've mentioned have been mostly addressed.

It's possible that you could use wp_scripts and wp_styles to get the enqueued scripts and styles. It might not be that simple, but that looks like a good starting point!

Exactly what I had in mind. I've tried this already, however there still seems to be an issue with [playlist]. I've described it in the "Known Issues" section in the PR description.

[playlist] shortcode was previously not rendering in the preview due to some missing scripts. This commit fixes this issue. Try using [playlist] in the shortcode block and enjoy the preview.

niranjan-uma-shankar · 2018-01-28T17:37:05Z

I've fixed the [playlist] preview issue in my latest commit. Woot!

niranjan-uma-shankar · 2018-01-30T11:59:24Z

Sorry for the noob question, since I'm new to jest. Requesting some pointers to fix the Travis failure - thought it would be quicker to resolve this with help from experienced jest users.

I understand it is caused, in the tests, by rendering a component wrapped in connect(). As recommended here, a way to fix this would be to export an unconnected component.
How would I do it in this context? I notice here that the block's edit key's value is used to render, and I am cautious not to edit a parent used by several other components.

UPDATE : I've fixed this.

The jest tests failed since I was using a connected component in Shortcode, which caused the tests to act up. Followed the recommended fix and updated the snapshot to reflect the new UI for the shortcode block.

The shortcode preview result is cached using Transients, for quicker render times and avoiding processing overheads

Empty results are possible if the shortcode is given invalid attributes. In this case, we needn't unnecessarily store the empty string shortcode in transients, so I've added a check to take care of this scenario.

If shortcode output is empty, I've added a flow wherein we exit from the API immediately, and avoid some extra processing which won't apply to empty shortcode output data.

youknowriad · 2018-02-05T07:30:47Z

blocks/library/shortcode/index.js

+import { addQueryArgs } from '@wordpress/url';
+import BlockControls from '../../block-controls';
+import { getCurrentPostId } from '../../../editor/store/selectors';
+import { connect } from 'react-redux';


Might be good to separate the external, internal and WordPress dependencies like we do in other files

Agree, I will separate them out. Thanks for bringing to my attention.

youknowriad · 2018-02-05T07:33:14Z

blocks/library/shortcode/index.js


 /**
 * Internal dependencies
 */
 import './editor.scss';
 import PlainText from '../../plain-text';

+export class Shortcode extends Component {


Having a hard time figuring out everything here :) do you think we can separate the rendering of the preview into a separate component ShortcodePreview taking the content and the post id as a prop and showing the preview? It could use withAPIData to fetch the preview.

Thanks for the feedback @youknowriad . Sure, will try to refactor this piece to a separate component.
I was just looking up the withAPIData component, and at how to send query params ( since I need to send the shortcode and postId to the REST API). The latest-posts block , which uses this, seems to have a problem. I get a 404 error. Do you see this on your build too? (I have rebased against master to have the latest version). Attaching below screenshot of what my Chrome network dev tool sees:

I'm assuming this to not be a bug with withAPIData component, but nevertheless bringing to your attention to confirm

Sometimes issues like that happen if you use "plain links" for permalinks, try using something else.

@youknowriad Yes, issue was with plain links. Changed permalinks in settings and it works now.
I've refactored code, as suggested by you to use a ShortcodePreview component. Also, I've addressed the other feedback comments you'd shared. Thanks for your time in reviewing and sharing feedback.

Also, just a note - all tests have passed in my local system. Travis is having issues currently (I was told in #core-editor slack), hence the failure.

UPDATE : Travis is back, all checks passed

youknowriad · 2018-02-05T07:35:33Z

components/sandbox/index.js

@@ -162,8 +174,11 @@ export default class Sandbox extends Component {
 					<style dangerouslySetInnerHTML={ { __html: style } } />
 				</head>
 				<body data-resizable-iframe-connected="data-resizable-iframe-connected" className={ this.props.type }>
-					<div dangerouslySetInnerHTML={ { __html: this.props.html } } />
+					<div id="content" dangerouslySetInnerHTML={ { __html: this.props.html } } />


why do we need a separate div for the html, js and styles can't we just concat everything when passing the html prop?

Added a couple of messages in cases when null values are sent as args to the API.

Lewiscowles1986 · 2018-02-25T14:28:52Z

lib/class-wp-rest-shortcodes-controller.php

+		wp_footer();
+		$footer_scripts_styles = ob_get_clean();
+
+		// Check if shortcode is returning a video. The video type will be used by the frontend to maintain 16:9 aspect ratio.


I don't know why this is still here, but it doesn't seem to offer anything over applying the the_content filter. Can you tell me about why you prefer to retain this method and having a "type"?

@Lewiscowles1986 Are you referring to why I'm determining the type? I'm finding out the type since sandboxing video content sets the height, as mentioned here. I see that you're always returning the type value as html - so this condition is never met?

That condition isn't for shortcode component though. You're correct that condition of the sandbox component isn't met, because there is no certainty the front-end will enforce a 16:9 aspect ratio. For me it's all about trying to get as close to 1:1 expectations

Lewiscowles1986 · 2018-02-25T14:31:21Z

lib/class-wp-rest-shortcodes-controller.php

-			'style' => $style,
-			'js'    => $js,
+			'html'                  => $output,
+			'head_scripts_styles'   => $head_scripts_styles,


this is great but the schema at the bottom of the file is still reflecting style and js names

I've updated the schema, thanks.

Lewiscowles1986 · 2018-02-25T14:53:56Z

phpunit/class-rest-shortcodes-controller-test.php

-					'type'  => 'html',
-					'style' => '',
-					'js'    => '',
+					'html' => '<p>\xe2\x82\xa1</p>',


As with the plain-english version, if the text sent in matches the text output with autop applied (I tried removing the filter from the_content, damn thing is like the flu it wouldn't go away), then should that be an error-case?

Lewiscowles1986 · 2018-02-25T17:59:39Z

I've found a problem...

The rest controller should only call do_shortcode, not any other filters. The reason is simple. The shortcode block does not handle oEmbed content outside of explicit shortcode when embedding in a page.

Unless the standard WP core / Gutenberg shortcode block outputs url's with an oEmbed provider in a shortcode block, simply calling apply_filters( 'the_content', $shortcode ); sets the user up for disappointment and confusion and would likely result in more support requests from confused users.

ehg · 2018-02-26T11:32:33Z

lib/class-wp-rest-shortcodes-controller.php

+				$embed_request['url'] = $matches[5][0];
+				$embed_response       = rest_do_request( $embed_request );
+				if ( $embed_response->is_error() ) {
+					// Convert to a WP_Error object.


The error handling here won't work. Also, why are we wp_die()ing?

TBF I had hoped that the REST API handled this as I glanced at it yesterday

ehg · 2018-02-26T11:34:51Z

Should we be handling links within previewed shortcodes better? For example, I get odd behaviour with the [gallery] shortcode.

Lewiscowles1986 · 2018-02-26T11:43:01Z

@ehg could you share an example of what you mean?

westonruter · 2018-02-26T23:19:51Z

@Lewiscowles1986:

The rest controller should only call do_shortcode, not any other filters. The reason is simple. The shortcode block does not handle oEmbed content outside of explicit shortcode when embedding in a page.

Yeah, that's probably a good point. However, it's important that the embed shortcode be registered. In other words:

add_shortcode( 'embed', array( $wp_embed, 'shortcode' ) );
$content = do_shortcodes( … );
// Restore shortcodes.

The block could short-circuit trying to send off a request (and show an error) if it doesn't contain a valid/recognized shortcode in the first place.

westonruter · 2018-02-26T23:22:07Z

Or rather, per WP_Embed::run_shortcode() it is important for the embed shortcode to be run first. So I think this is what we need:

$content = $wp_embed->run_shortcode( $content );
$content = do_shortcode( $content );

Lewiscowles1986 · 2018-02-27T08:20:08Z

@westonruter in my fork I am running embed or do_shortcode (although I see how it should be both). I don't see that this changes things regarding "type". (I feel like shortcodes should always return "html")

Lewiscowles1986 · 2018-02-27T17:28:33Z

PoC shortcode preview re-factor into component without need for connect
https://github.com/Lewiscowles1986/gutenberg/blob/0fc2c6a17905362c3af15761707d0aac732f14da/blocks/library/shortcode/preview.js

matches blocks/library/embed/index.js use of window.fetch instead of withAPIData
inspired to use componentDidMount from https://github.com/11online/react-wp-plugin-demo/blob/master/wp-plugin/src/App.js instead of connect

Lewiscowles1986 · 2018-02-27T17:33:41Z

PoC shortcode suggested edits

handle embed and regular shortcodes

https://github.com/Lewiscowles1986/gutenberg/blob/0fc2c6a17905362c3af15761707d0aac732f14da/lib/class-wp-rest-shortcodes-controller.php#L106-L112

handle content not changing (shortcode invalid?)

https://github.com/Lewiscowles1986/gutenberg/blob/0fc2c6a17905362c3af15761707d0aac732f14da/lib/class-wp-rest-shortcodes-controller.php#L114-L117

Known misses

previously valid shortcodes would have no effects and return error message from REST endpoint
- should this be returning http 200?
- should this be preventing save?
static content is permitted surrounding and in-between shortcodes (not sure if this can be fixed in a way that doesn't leave more problems than it solves)

Lewiscowles1986 · 2018-02-27T17:35:03Z

Everything else here seems a matter of opinion, although I'd still like either preview to be default or a way found that enables me to write a plugin that triggers that behaviour (I don't care about the tabs)

Lewiscowles1986 · 2018-02-27T17:37:19Z

Suggestion

Preview to be wrapped in a div position: relative; and contain a child element position: absolute; top: 0; left: 0; width: 100%; height: 100% to prevent users interacting with the block contents. Perhaps another place to put controls overlaid over output shortcode content. (window vs interactive UI) pen

westonruter · 2018-03-12T16:30:37Z

lib/class-wp-rest-shortcodes-controller.php

+	public function __construct() {
+		// @codingStandardsIgnoreLine - PHPCS mistakes $this->namespace for the namespace keyword
+		$this->namespace = 'gutenberg/v1';
+		$this->rest_base = 'shortcodes';


I think this would be better named shortcode-renderer

westonruter · 2018-03-12T16:33:11Z

lib/class-wp-rest-shortcodes-controller.php

+	 * @return true|WP_Error True if the request has read access, WP_Error object otherwise.
+	 */
+	public function get_shortcode_output_permissions_check( $request ) {
+		if ( ! current_user_can( 'edit_posts' ) ) {


If $args['postId'] is not empty, then it should also check if current_user_can( 'edit_post', $args['postId'] ).

westonruter · 2018-03-12T16:38:12Z

lib/class-wp-rest-shortcodes-controller.php

+		$output                = '';
+		$args                  = $request->get_params();
+		$post                  = isset( $args['postId'] ) ? get_post( $args['postId'] ) : null;
+		$shortcode             = isset( $args['shortcode'] ) ? trim( $args['shortcode'] ) : '';


These args don't seem to be defined in the schema, so they aren't getting sanitized/validated. The postId should be validated to be an integer and refer to an existing post. The shortcode should be validated to actually only contain a shortcode.

westonruter · 2018-03-12T16:41:48Z

lib/class-wp-rest-shortcodes-controller.php

+					'type'        => 'string',
+					'required'    => true,
+				),
+				'head_scripts_styles'   => array(


Technically this will contain not just scripts and styles but anything that is output by wp_head (and wp_footer) below. I think this would be better named just html_head and html_footer, and maybe html_body instead of html.

danielbachhuber · 2018-05-09T13:39:41Z

Thanks again for your work on this, @niranjan-uma-shankar. Closing per #1054 (comment)

niranjan-uma-shankar · 2018-05-09T19:56:17Z

Thanks @danielbachhuber and everyone else who helped review this PR.

Lewiscowles1986 · 2018-05-12T22:56:44Z

As this is now closed, I've added this to a repo so people that want to use my offshoot (linked above) can still use shortcodes with preview in Gutenberg https://github.com/CODESIGN2/gutenberg-shortcode-preview-block/releases

System Administrator and others added 13 commits January 25, 2018 04:49

eslint fixes and reordering for better readibility and conforming to …

8cec7fc

…standards

Minor changes to the title and code comments to change language to sh…

9fc96f8

…ortcode

Post ID is now retrieved from redux store, instead of from GET params

00abaca

Previously, the post ID was fetched by processing the post's URL and parsing the post ID GET parameter. Now, a better approach is taken by reading the post ID value from the redux store.

Injects custom css and js that might be needed by iframe

3646f61

In this revision, custom css and js files are injected as props to the iframe sandbox. Custom css and js is needed in certain cases, for example, [gallery] shortcode needs the parent theme style.css.

Merge remote-tracking branch 'upstream/master'

3e5c6b4

fixes warnings

80800ed

fixes warnings

63549b4

fixes warnings

9997daf

fixes warnings

4095bca

fixes warnings

aa57071

Bug fix : [playlist] shortcode now renders in the preview

0e71158

[playlist] shortcode was previously not rendering in the preview due to some missing scripts. This commit fixes this issue. Try using [playlist] in the shortcode block and enjoy the preview.

System Administrator and others added 9 commits January 30, 2018 20:06

Fixes jest Snapshot which caused the tests to fail

4871b73

The jest tests failed since I was using a connected component in Shortcode, which caused the tests to act up. Followed the recommended fix and updated the snapshot to reflect the new UI for the shortcode block.

Merge remote-tracking branch 'upstream/master'

7f00de6

Enhancement: Implements caching of results

a6d250a

The shortcode preview result is cached using Transients, for quicker render times and avoiding processing overheads

lint fixes

3a55c5a

Fix: Don't save in transients if result is empty

7fdad34

Empty results are possible if the shortcode is given invalid attributes. In this case, we needn't unnecessarily store the empty string shortcode in transients, so I've added a check to take care of this scenario.

Fix: Handles better empty shortcode output scenario

a486b44

If shortcode output is empty, I've added a flow wherein we exit from the API immediately, and avoid some extra processing which won't apply to empty shortcode output data.

Minor fix: Translatable function for a string was missing, fixed it

c9f11b5

Added some null checks

5bf4ea3

Unit tests for the Shortcode preview rendering REST API.

861dd1c

youknowriad reviewed Feb 5, 2018

View reviewed changes

Added some error messages

bc9a96a

Added a couple of messages in cases when null values are sent as args to the API.

Lewiscowles1986 reviewed Feb 25, 2018

View reviewed changes

Undo logging changes

49b2e85

Lewiscowles1986 reviewed Feb 25, 2018

View reviewed changes

Updated schema, and fixed lint

4595fb4

ehg reviewed Feb 26, 2018

View reviewed changes

Lewiscowles1986 mentioned this pull request Feb 27, 2018

#4710 do not merge #5287

Closed

3 tasks

Lewiscowles1986 mentioned this pull request Feb 27, 2018

New shoes Lewiscowles1986/gutenberg#1

Closed

3 tasks

Reverting API changes to do_shortcode/run_shortcode method

777b304

niranjan-uma-shankar mentioned this pull request Mar 6, 2018

Make any iframe embed responsive in the editor #5333

Merged

westonruter reviewed Mar 12, 2018

View reviewed changes

Lewiscowles1986 mentioned this pull request Mar 14, 2018

Add Server Side Render component. #5602

Merged

3 tasks

niranjan-uma-shankar added 3 commits April 15, 2018 17:01

Merge

80ae62a

Merge

d81a308

Merge remote-tracking branch 'upstream/master'

12d83a0

danielbachhuber closed this May 9, 2018

Lewiscowles1986 mentioned this pull request May 10, 2018

Add widget_posts_args filter from Recent Posts widget to also apply in Latest Posts block #4808

Closed

3 tasks

Enhance the shortcode block to support previewing of shortcodes #4710

Enhance the shortcode block to support previewing of shortcodes #4710

Conversation

niranjan-uma-shankar commented Jan 27, 2018 • edited Loading

Description

Summary of changes

How to test

Screenshot

Known issue

niranjan-uma-shankar commented Jan 28, 2018

niranjan-uma-shankar commented Jan 28, 2018

niranjan-uma-shankar commented Jan 30, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

niranjan-uma-shankar Feb 5, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

niranjan-uma-shankar Feb 6, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Lewiscowles1986 Feb 25, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Lewiscowles1986 commented Feb 25, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ehg commented Feb 26, 2018

Lewiscowles1986 commented Feb 26, 2018

westonruter commented Feb 26, 2018

westonruter commented Feb 26, 2018

Lewiscowles1986 commented Feb 27, 2018

Lewiscowles1986 commented Feb 27, 2018 • edited Loading

Lewiscowles1986 commented Feb 27, 2018

handle embed and regular shortcodes

handle content not changing (shortcode invalid?)

Known misses

Lewiscowles1986 commented Feb 27, 2018

Lewiscowles1986 commented Feb 27, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

danielbachhuber commented May 9, 2018

niranjan-uma-shankar commented May 9, 2018

Lewiscowles1986 commented May 12, 2018 • edited Loading

niranjan-uma-shankar commented Jan 27, 2018 •

edited

Loading

niranjan-uma-shankar commented Jan 30, 2018 •

edited

Loading

niranjan-uma-shankar Feb 5, 2018 •

edited

Loading

niranjan-uma-shankar Feb 6, 2018 •

edited

Loading

Lewiscowles1986 Feb 25, 2018 •

edited

Loading

Lewiscowles1986 commented Feb 25, 2018 •

edited

Loading

Lewiscowles1986 commented Feb 27, 2018 •

edited

Loading

Lewiscowles1986 commented Feb 27, 2018 •

edited

Loading

Lewiscowles1986 commented May 12, 2018 •

edited

Loading