Blueprints: Explore switching to the PHP implementation

[adamziel]

What is this PR doing?

Explores using the blueprints.phar PHP library in Playground.

For more context see:

• Blueprints: Transition from a TypeScript library to a PHP library #1025
• https://github.com/WordPress/blueprints

For now, the PHP library works nicely in local CLI but throws the following error in Playground:

Work explored in this PR

Arbitrary http:// and https:// network calls

The Blueprint library isn't currently able to make any network calls.

There are two problems:

1. allow_url_fopen is set to 0. This is easy to change.
2. The web version of Playground doesn't support the http:// and https:// stream wrappers.

There are two possible solutions:

1. Implement a Fetch HTTP transport, similarly to what we do in WordPress. This is easy and mostly reliable, but doesn't generalize to other PHP code which means we'll need a transport every time we want to support another request-issuing library.
2. Add support for the http:// and https:// stream wrappers. This is challenging! All the network traffic originating in the browser must eventually be funneled through fetch(). However, HTTPS communication requires exchanging data packets through a raw socket. Perhaps Playground could run a MITM attack on the WebAssembly PHP instance and rewrite those decrypted raw bytes as fetch() input?

Update I wend for HTTPS termination and rewriting and it worked! It's still a super rough MVP, but it works! It should also remove the need for a custom Fetch transport for WordPress and also open the door to supporting libcurl.

proc_open improvemens

PHP subprocesses

#1031 is merged into this PR

cwd and env variables in proc_open

The Blueprints library leans on running PHP subprocesses in a specific working directory and passing data through env variables, e.g. like DOCROOT=/wordpress.

Exit code propagation

The trunk version of Playground doesn't correctly communicate the proc_open exit code to the PHP code. The following update to phpwasm-emscripten-library.js solves the problem:

- HEAPU8[exitCodePtr] =  PHPWASM.child_proc_by_pid[pid].exitCode;
- wakeUp(0);
+ HEAPU32[exitCodePtr >> 2] =  PHPWASM.child_proc_by_pid[pid].exitCode;
+ wakeUp(pid);

Also, proc_get_status() now returns the correct exitcode and stopped values.

Fixing a race condition in the startup sequence

If the PHP web worker thread takes too long to start up, e.g. because it synchronously generates SSL keys, the remote.html will correctly resolve the client.isReady() promise, but the client object on the topmost HTML page will not. That results in a blank page in this demo, and can result in an always-empty empty progress bar on https://playground.wordpress.net/.

Streaming ZIPs

Stream-unzipping gutenberg.zip in the browser yields the following error:

Error: PHP.run() failed with exit code 255 and the following output: PHP Fatal error:  Out of memory (allocated 16777216 bytes) (tried to allocate 131072 bytes) in phar:///wordpress/blueprints.phar/src/WordPress/Zip/ZipStreamReader.php on line 207

Other rough notes

• Worker thread startup race condition when ssl keys are generated
• Need to HTTPS certificates generate when WebSocket starts
• Box.phar "check-requirements": false, breaks HTTP requests in Playground :o
• Out of memory error when fed real 40MB WordPress.zip – is it related to chunking/buffering/blocking calls to onData()?

[adamziel]

This Blueprint works in the latest version of this branch AND in native PHP! Internally, it relies on concurrent file downloads and WP-CLI! #1053 was the crucial bit to make the network transfer work.

$blueprint = BlueprintBuilder::create()
	->withFile( 'wordpress.txt', (new UrlResource())->setUrl('https://downloads.wordpress.org/plugin/hello-dolly.zip') )
	->withSiteOptions( [
		'blogname' => 'My Playground Blog',
	] )
	->withWpConfigConstants( [
		'WP_DEBUG'         => true,
		'WP_DEBUG_LOG'     => true,
		'WP_DEBUG_DISPLAY' => true,
		'WP_CACHE'         => true,
	] )
	->withPlugins( [
		'https://downloads.wordpress.org/plugin/hello-dolly.zip',
		'https://downloads.wordpress.org/plugin/classic-editor.zip',
		'https://downloads.wordpress.org/plugin/gutenberg.17.7.0.zip',
	] )
	->withTheme( 'https://downloads.wordpress.org/theme/pendant.zip' )
	->withContent( 'https://raw.githubusercontent.com/WordPress/theme-test-data/master/themeunittestdata.wordpress.xml' )
	->andRunSQL( <<<'SQL'
		CREATE TABLE tmp_table ( id INT );
		INSERT INTO tmp_table VALUES (1);
		INSERT INTO tmp_table VALUES (2);
		SQL
	)
	->withFile( 'wordpress.txt', 'Data' )
	->toBlueprint()

[adamziel]

All the necessary Playground enhancements are duct-taped together in this PR, the next steps are to split them out into separate PRs, merge into core, and rebase this PR until it only ships the showcase page.

[adamziel]

All the PHP.wasm updates required for the PHP Blueprints to run are now merged 🎉 The demo page is explored in PR #1070. The only remaining part of this PR that did not get merged into Playground core is a generic HTTPS -> network bridge that doesn't require implementing special transport classes in PHP. Let's split it out into a new PR before closing this one.

[adamziel]

For posterity, the HTTPS proxy explored in this PR requires the following change in the PHP library UrlSource class:

		$response = $this->client->request( 'GET', $url, [
			'on_progress'   => function ( int $dlNow, int $dlSize, array $info ) use ( $url ): void {
				$this->events->dispatch( new ProgressEvent(
					$url,
					$dlNow,
					$dlSize
				) );
			},
+			//        We use a fake SSL server in there to MITM the HTTPS requests
+			//        and funnel them through fetch() – and fetch() handles HTTPS
+			//  	  security for us.
+			'verify_host'   => false,
+			'verify_peer'   => false,
+			'crypto_method' => \STREAM_CRYPTO_METHOD_TLSv1_0_CLIENT,
		] );

[adamziel]

Also, response buffering in UrlSource seems to cause OOM errors

		$onChunk = function ( $chunk ) use ( $url, $response, $stream ) {
			// Handle response caching
			static $bufferedChunks = [];
			$bufferedChunks[] = $chunk;
			if ( feof( $stream ) ) {
				$this->cache->set( $url, implode( '', $bufferedChunks ) );
				$bufferedChunks = [];
			}
		};

[adamziel]

All parts of this PR except #1093 are now merged into trunk. Let me close it then, and let's explore the native HTTPS support in #1093.