Sharing Filesystem between multiple PHP instances

[adamziel]

Blueprints as a PHP library depends on a sharing the filesystem between two PHP instances.

Many tasks must be delegated to a PHP sub-process which makes no sense if that sub-process has no access to WordPress files seen by the main process.

Imagine the following scenario:

<?php proc_open(['php', 'activate_theme.php', 'pendant']);

It could be handled by PHP.wasm as follows:

php1.setSpawnHandler(
	createSpawnHandler(async function (args, processApi) {
		const php2 = await WebPHP.load();
		const result = await php2.run({
			scriptPath: args[1]
		});
		processApi.exit(result.exitCode);
	})
);

If, however, php2 acts on a separate filesystem, the pendant theme won't be activated from the perspective of php1.

This issue is about Web Browser. In Node.js, NODEFS solves this problem

wp-now uses the filesystem of the device it runs on via the NODEFS Emscripten API.

Reusing the same filesystem

PHP.wasm uses MEMFS by default

A newly created PHP instance handles all filesystem operations using an in-memory filesystem implementation called MEMFS. MEMFS keeps track of the files using JavaScript objects. It also contains hardcoded references to HEAP and FS of the PHP instance it lives in

Reusing MEMFS seems like a non-starter

Sharing MEMFS between two PHP instances seems extremely difficult. The hardcoded heap and FS references makes it difficult to bind the same MEMFS to two PHP instances. Perhaps it could be done with deep refactoring, but I worry we'd quickly run into a problem of reusing heap, but not the static memory or stack, between PHP instances. It doesn't seem to be worth it.

IDBFS is too slow

It takes a few minutes to read the WordPress files from IDBFS into MEMFS. It's just too slow for this amount of I/O.

Conceptually, OPFS could work. Unfortunately, the Emscripten OPFS backend crashes

Ditching MEMFS and relying on OPFS would enable all the PHP instances to act on the same underlying files. Emscripten has new and undocumented support for OPFS. I explored it in this PR and, unfortunately, couldn't get it to work without crashing:

• Experiment: Build PHP with OPFS support #1030

Synchronizing two filesystems

If we can't easily share the same filesystem, synchronizing two distinct filesystems is the next best approach.

Overwriting the entire filesystem

Whenever the child process yields to the event loop, we could overwrite the main process's filesystem with all the files from the child's filesystem.

I can only think of two issues with this approach:

Safety – could it affect the main FS in an unsafe way that would not happen with concurrent writes to a shared Filesystem?

I'm not sure, but intuitively, I want to say it's safe. There is no real concurrency involved – this could work in a single JavaScript worker on a single event loop:

1. The main process pauses, the child process starts
2. The child process does some work, optionally pipes stdout data
3. The main process optionally resumes to process the stdout data before pausing again
4. The child process eventually finishes
5. The main process continues

Because everything happens in order, replaying the filesystem operations intuitively seem safe to me. Or at least not less safe than running both processes in parallel on my Mac.

Is there a flaw in this reasoning?

Speed – would overwriting the entire filesystem take ages?

My intuition says yes, but rotatedPHP performs exactly this kind of filesystem overwrite and it's barely noticeable. If, however, the speed turns out to be a problem after all, we could turn to the next approach on the list.

Replaying the filesystem operations

Playground supports synchronizing two Playground instances by journaling and replaying the MEMFS operations – see the (demo](https://playground.wordpress.net/demos/sync.html).

This PR explores syncing and replaying FS operations:

• Prototype: Spawning PHP sub-processes in Web Workers #1031

Related points

Emscripten supports PThreads and WASM Workers. They seem tempting at first, but they aren't a silver bullet. The underlying implementation is just built with web workers and SharedArrayBuffer, neither of which solves our problem here.

We could potentially use the same SharedArrayBuffer in both PHP instances to keep track of MEMFS files. However, and I'm making a few guesses that could be wrong, we'd end up also sharing the heap, the stack, and the global variables and locks as well. That would make it the same as just calling run() on the current PHP instance, which we can't do. Please tell me if wrong about anything here!

[adamziel]

This PR explores syncing and replaying FS operations:

• Prototype: Spawning PHP sub-processes in Web Workers #1031

[adamziel]

Actually, Emscripten's native PROXYFS provides this exact feature 🎉 This means we can have two (or more!) Emscripten modules acting on the same filesystem.

// Module 2 can use the path "/fs1" to access and modify Module 1's filesystem
module2.FS.mkdir("/fs1");
module2.FS.mount(module2.PROXYFS, {
    root: "/",
    fs: module1.FS
}, "/fs1");