HTML API: Return elements pushed and popped rather than tags read.

[dmsnell]

Trac ticket: Core-61348

Summary

Creates virtual nodes when pushing to and popping from the stack of open elements. It's these nodes that are returned by next_tag(), while subclassed methods intercept tag information, all within the HTML Processor.

Splitting time!

• Add get_current_depth() to return the depth of the currently-matched element in the stack of open elements. This needs to account for marker and other not-yet-implemented items in the stack.
• Add expects_closer() or similar function to indicate if the currently-matched element needs or expects a closing element.

Questions

• What do we do with void tags?
  • There are tradeoffs in how to represent tags without a closing tag (be they void elements or self-closing elements within foreign content or special atomic elements like SCRIPT). We can represent a unique open and close tag/event for them, or only expose the opening.
    • In exposing an open and a close there's a very straightforward and universal model of sequential open and close events for every tag.
    • It's weird to have a closing tag for an IMG.
  • I think that the value in having a close tag is outweighed by the awkwardness of it. For situations where we're doing things like tracking depth, we should rely on an internal method like get_depth() from the HTML API itself instead of exporting that HTML nuance onto the caller.

Related work

• WIP: HTML API: Add function to balance tags ala force_balance_tags() #5562
• WIP: Introduce class for sourcing block attributes from HTML gutenberg#46345

Examples

Convert to Markdown	Output

Normalize HTML	Output

cc: @sirreal

[github-actions]

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

• The Plugin and Theme Directories cannot be accessed within Playground.
• All changes will be lost when closing a tab with a Playground instance.
• All changes will be lost when refreshing the page.
• A fresh instance is created each time the link below is clicked.
• Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
  it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

[dmsnell]

@sirreal it looks like we're failing tests after seeking, which I believe might be because when jumping back to the start of the document, we wipe out the context node.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props dmsnell, audrasjb, jonsurrell.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[sirreal]

Should this also use $this?

Suggested change

	if ( ! parent::is_tag_closer() || $visit_closers ) {
	if ( ! $this->is_tag_closer() || $visit_closers ) {

[dmsnell]

Yes. Can you concoct a test case to fail for it? If not, don't worry - we can just fix it.

[sirreal]

There's a regression here where this HTML:

<?wp-bit hey?>

This is a PI_NODE_LOOKALIKE. Before, get_tag() here would be wp-bit - corresponding to the PI target. After this change get_tag() returns #comment.

[sirreal]

This is what breaks the tag_name for PI lookalikes. The node_name is #comment, but it's a special comment type that has more handling in the Tag Processor:

wordpress-develop/src/wp-includes/html-api/class-wp-html-tag-processor.php

Lines 2640 to 2651 in fbb5020

	$tag_name = substr( $this->html, $this->tag_name_starts_at, $this->tag_name_length );
	if ( self::STATE_MATCHED_TAG === $this->parser_state ) {
	return strtoupper( $tag_name );
	}
	if (
	self::STATE_COMMENT === $this->parser_state &&
	self::COMMENT_AS_PI_NODE_LOOKALIKE === $this->get_comment_type()
	) {
	return $tag_name;
	}

A potential fix is to only return here if we have a "non-special" node:

Suggested change

	if ( isset( $this->current_element ) ) {
	return $this->current_element->token->node_name;
	}
	if ( isset( $this->current_element ) && '#' !== $this->current_element->token->node_name[0] ) {
	return $this->current_element->token->node_name;
	}

[sirreal]

The following HTML reaches a P tag at the wrong depth:

EDIT: This is pre-existing behavior.

redacted

<div></p>

$p = WP_HTML_Processor::create_fragment('<div></p>');
$p->next_token();
$p->next_token();
var_dump( $p->get_tag(), $p->get_current_depth(), $p->get_breadcrumbs() );

php shell code:1:
string(1) "P"
php shell code:1:
int(3)
php shell code:1:
array(3) {
  [0] =>
  string(4) "HTML"
  [1] =>
  string(4) "BODY"
  [2] =>
  string(3) "DIV"
}

[dmsnell]

Merged in [58304]
163c3fd

[westonruter]

Is there a reason this is specifically a Closure and isn't just more generally a callable?

[dmsnell]

Closures cannot be serialized or deserialized, meaning that there's no possible way to prep a database record with user input that sets something unexpected here.