Parsing: Use full parser in do_blocks with nested block support

[dmsnell]

Updates do_blocks() and gutenberg_render_block() so that we can support nested blocks inside of dynamic blocks. This replaces the use of the partial parser which extracts registered dynamic blocks with the full parser.

This change will allow dynamic blocks which contain nested blocks inside of them and it will pave the way for a filtering API to structurally process blocks.

The partial parser came about at a time before the default parser was written; it was faster than the spec parser and was a tradeoff to get dynamic blocks rendering. The default parser, however, has been fast enough for a while to run on page render and so this PR exists to finally get it into the pipeline.

Status

Iteration

The RecursiveIteratorIterator should provide us a way of performing the depth-first block traversal without risking a stack overflow. We keep rendered blocks on a stack and pop them off as we go.

Interestingly enough I did some testing and RecursiveIteratorIterator was a let-down. There's one important thing it does which is bypass the xdebug.max_nesting_level which limits recursion depth when xdebug is running; sadly it's much much slower than basic recursion.

With the RecursiveIteratorIterator we can continue to nest into blocks until we run out of memory but the runtime performance is exponential. With recursion we hit the same memory limit if xdebug is disabled but hit it usually around 100 if it is. That means that for massive posts we could run into unexpected failures when debugging if we go the recursion route.

What is the memory limit? In my testing back to PHP 5.6 on my local laptop it was at around 100,000 levels of nesting - this should be obviously sufficient - I can't imagine an editor showing that many levels of nesting. With the recursive version and a limit of 100 that's still almost equally viable as 100,000. I'd recommend we stick with this recursive approach then unless we get bug reports of failures when debugging, at which point we can discuss bringing back the RecursiveIteratorIterator version.

On the other hand I may simply have been writing the WP_Block_Tree_Iterator in a stupid way that was way too slow. It would be worth running an experiment where I make a custom tree iterator without using the SPL functions and see if the performance characteristics are similar.

Update

After running some tests I made ended up with a basic tree iterator and it hit hard performance issues long before the recursive version. I think that the memory churning of the stacks required to track nodes in the tree is far less efficient than using the built-in argument passing in functions and closures provided by PHP with the recursive method.

In other words, I am currently unable to come close to the recursive version in terms of performance. Does this matter with small nesting? Probably not. Should we then prefer the safe version that won't break with low max_nesting_depth? I don't know. Probably not since anything over 100 levels of nested blocks is somewhat of an insane construction.

Testing and Review

The patch needs testing, but this can be hard because we don't have too many deeply-nested blocks. It's best to try with things like columns.

👉 I would greatly appreciate some help adding the appropriate deprecation messages for get_dynamic_blocks_regex() and friends!

[georgestephanis]

encapsulated ^^ into a pr, with unit tests:

#11227

[youknowriad]

Moving out of 4.2 as we have to release soon.

[dmsnell]

Previous changes on this branch before the rebase…
4a4feba

[dmsnell]

@aduth if you look at the failing tests you can see that we're hitting trailing-newline issues because in the existing do_blocks() we strip [\r\n] after a block's closing comments but here we're not.

can you help me understand what our desired end-product is so I can implement it? or, if it doesn't matter, should I update the test fixtures?

if we look at the parser output we can see that these newlines end up as freeform blocks whose contents are exactly \n\n. from a WordPress standpoint I guess that could risk introducing autop paragraphs? from an HTML standpoint it's just extra meaningless whitespace.

[pento]

get_dynamic_blocks_regex() hasn't been merged to Core, but it's still being used by strip_dynamic_blocks().

Upon reflection, I don't think strip_dynamic_blocks() was the right solution for the problem (#7268) that it was originally intended to solve. Something more like what @chrisvanpatten described would probably give us results more in line with how wp_trim_excerpt() is intended to behave.

I'll follow up with a PR after this one is merged to explore that.

[pento]

👍🏻 Lovely, than you for wrangling this, @dmsnell!