Memory leak: add rotatedPHP to kill and recreate PHP instances after a certain number of requests

[adamziel]

Supersedes WordPress/playground-tools#110

Swaps the internal PHP runtime for a fresh one after a certain number of requests are handled.

Rationale

PHP and PHP extension have a memory leak. Each request leaves the memory a bit more fragmented and with a bit less available space than before. Eventually, new allocations start failing.

Rotating the PHP instance may seem like a workaround, but it's actually what PHP-FPM does natively:

Implementing this solution in core enables all downstream consumers of the PHP WASM package, like wp-now, to benefit.

How it works

Adds a rotatePHP function that keeps track of the BasePHP.run() calls and replaces the Emscripten runtime with a new one after a certain number of calls.

Example:

import { rotatePHPRuntime } from '@php-wasm/universal';

const loadRuntime = async () => await NodePHP.loadRuntime("8.0");

const php = new NodePHP(await loadRuntime(), {
	documentRoot: '/test-root',
});

await rotatePHPRuntime({
	php,
	recreateRuntime: loadRuntime,
	maxRequests: 50
});

// After 50 request() calls, the PHP runtime will be discarded and a new one will be created.
// It all happens internally. The caller doesn't know anything about this.

I started by porting the "Process Pool" idea (see the branch) from WordPress/playground-tools#110, but I realized that:

• The logic was complicated
• We only need to manage a single PHP instance, not a pool of them
• Worker thread is built to a single PHP instance and not architected to handle a pool

So I decided to simplify the logic and just manage a single PHP instance.

Other changes

• Implements BasePHP.hotSwapPHPRuntime() that opens the door to switching PHP versions dynamically and without a full page reload.
• Makes the BasePHP.semaphore property public to enable conflict-free runtime swaps
• Adds the runtime.initialized and runtime.beforedestroy events for OPFS handlers to re-bind on runtime swap.
• Tiny unrelated change: Migrates the OPFS handler from monkey-patching the php.run method to relying on event listeners. The rotatePHP function went through a similar journey and I updated that other code path while I was on it.

Testing instructions

confirm the CI checks pass

CC @dmsnell

[adamziel]

This approach unlocks hotswapping PHP versions without forcing a full page reload – just rotate to PHP 8.2, then 8.1 etc.

[dmsnell]

this looks more robust than the previous approach, and I like how this swaps out a single runtime vs. pre-reserving a pool of workers.

how are we deciding on the number of max requests? would it be helpful to shorten that list? would it be helpful to analyze total memory use and make it depend on that?

[dmsnell]

what's the point of these lines?

[adamziel]

These run() calls are meant to trigger the internal PHP runtime rotation. I just replaced the rotate() function with something more readable:

		// Rotate the PHP runtime
		await php.run({ code: `` });

With no "echo" involved, the intention is hopefully less obscured

[adamziel]

how are we deciding on the number of max requests? would it be helpful to shorten that list? would it be helpful to analyze total memory use and make it depend on that?

I just added a comment to answer these questions in the code. 400 is an arbitrary number I pulled out of a hat. It seems to work well in practice on my setup in that:

• The memory-related crash is not happening anymore
• The small slowdown required to initialize the new runtime and copy the MEMFS files is rare enough that it goes mostly unnoticed.

I realize there might be ways of using Playground that will leave the memory extremely fragmented after less than 400 requests. I don't know whether they exist or how to identify them so you're right – ideally the rotation logic would be based on the amount of free, unfragmented memory. I'm yet to find a fast method of keeping track of that – even a naive approach of calculating the null bytes takes a few blocking seconds. I think I'll leave that hardcoded 400 in there for now with a mental note to keep exploring should the memory issue be reported again.

Also CC @bgrgicak

[adamziel]

For posterity – it would be so, so lovely to directly reuse the oldFS object without going through the copying here.

[bgrgicak]

It's mindblowing that we can swap php versions on the fly 🚀

[bgrgicak]

TIL 30_000 === 30000

[dmsnell]

we can swap php versions on the fly

idea: randomly pick the PHP versions on swap just to mess with people 🙃