Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A boot() method to explicitly initialize the PHP worker #1669

Merged
merged 19 commits into from
Aug 19, 2024

Conversation

adamziel
Copy link
Collaborator

@adamziel adamziel commented Aug 1, 2024

Motivation for the change, related issues

Refactors the remote.html and worker-thread.ts boot flows.

  • Before this PR, both files passed configuration using query arguments and relied on implicit logic surrounding the endpoint class
  • After this PR, both API Endpoint classes expose a boot() method that is strongly typed, accepts the configuration values, and explicitly orchestrates all the boot logic when it is called.

This work may eventually enable:

  • A reusable, isomorphic Worker implementation that can be reused in the browser and in a local CLI setup.
  • Exposing bootWordPress() on a worker instance.
  • Hot-swapping the PHP runtime and WordPress instance (e.g. by killing the current worker and starting a new one).
  • Spawning new workers, not just PHP class instances, via PHPProcessManager.

Related to #1398

Follow-up work

  • Explore spawning multiple workers for handling requests. This might also require:
    • Exposing the full boot protocol (bootWordPress()) on a worker instance instead of the simplified boot() method.
    • Load balancer across all workers handling the same site instance.
    • An OPFS Emscripten filesystem backend so that each worker can see the changes made by other workers. This is a potential blocker as Emscripten didn't seem to have a good way of doing that a few months ago.

Testing Instructions (or ideally a Blueprint)

  • Confirm the tests are all green
  • Carefully review the changes, point out anything that seems off

cc @brandonpayton @bgrgicak

@adamziel adamziel marked this pull request as ready for review August 1, 2024 23:53
@adamziel adamziel changed the title Boot: worker.boot() method to standardize spinning new PHP workers Boot: Explicitly start PHP workers using a boot() method Aug 1, 2024
@adamziel
Copy link
Collaborator Author

adamziel commented Aug 2, 2024

This is ready for review now.

Also, I wonder if bringing in files from git repositories could be expressed as a "read-only mount" 🤔 This would neatly interop with the CLI version.

@adamziel adamziel changed the title Boot: Explicitly start PHP workers using a boot() method Boot: Explicitly initialize PHP workers using a boot() method Aug 2, 2024
@adamziel adamziel changed the title Boot: Explicitly initialize PHP workers using a boot() method A boot() method to explicitly initialize the PHP worker Aug 2, 2024
@adamziel adamziel self-assigned this Aug 2, 2024
@adamziel
Copy link
Collaborator Author

adamziel commented Aug 2, 2024

I know it's a large one and might be difficult to review. There's a few things I could extract, but unfortunately most of it is an atomic change. If nothing stands out, we could try deploying that on Monday with full readiness to rollback.

Copy link
Member

@brandonpayton brandonpayton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not done reviewing but wanted to share progress so far because there is a blocker.

The Good:

  • The name improvements in this PR help me so much. I started leaving comments about that, but there were so many that I eventually stopped commenting on it.
  • Switching to explicit boot calls seems like a big improvement to clarity and function.

The Bad:

Copy link
Collaborator

@bgrgicak bgrgicak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a big PR 😅

+1 to what @brandonpayton mentioned. I like how the code is now easier to follow and better named.

This feels like a good, timing for a refactor, we added a lot of features to the boot flow recently and some changes were all over the place.

@brandonpayton
Copy link
Member

I pushed 4d1a539 which passes around OPFS paths rather than FileSystemDirectoryHandles which due to the Safari structuredClone() not supporting that.

It looks like this may fix #1239 because Safari is loading Playground with browser storage locally after this change.

@brandonpayton
Copy link
Member

There are some merge conflicts with trunk, so I'll look at those next.

@brandonpayton brandonpayton requested a review from a team as a code owner August 15, 2024 01:08
@brandonpayton
Copy link
Member

I missed some thing in the original switch from passing OPFS handles to passing OPFS paths, and those should be fixed now.

@brandonpayton
Copy link
Member

I am currently seeing a bug with creating a new site on device storage:

index.tsx:139 Uncaught (in promise) DOMException: Unable to resolve path of OPFS directory handle.

Discovered while testing for this review comment:
https://github.com/WordPress/wordpress-playground/pull/1669/files#r1705279446

Looking into it.

@brandonpayton
Copy link
Member

It looks like I was mistaken about the "fix" for this PR because we aren't just passing around opfs paths. FileSystemDirectoryHandles are also used for local FS access, so we cannot just pass paths around and assume they are for OPFS.

Since FileSystemDirectoryHandle instances cannot be transferred via postMessage() in Safari, we need a different solution.

Continuing to look into this.

@brandonpayton
Copy link
Member

One thing to remember is that Safari doesn't support direct local FS access via showDirectoryPicker(). So we could consider passing FileSystemDirectoryHandles for the local FS case because local FS access doesn't work in Safari anyway.

@brandonpayton
Copy link
Member

brandonpayton commented Aug 16, 2024

I worked on adjusting this to use the following types:

export type MountDevice =
	| {
		type: 'opfs';
		path: string;
	  }
	| {
		type: 'local-fs';
		// Since we have to prompt the user to get the FS handle,
		// we cannot simply save a path and recreate the handle
		// like we can do for OPFS. So we have to pass the handle between threads.
		handle: FileSystemDirectoryHandle;
	  };

export interface MountDescriptor {
	mountpoint: string;
	device: MountDevice;
}

Safari doesn't support structuredClone() for FileSystemHandle or the platform features we use to interact with the local device filesystem. Since we don't support local-fs sites on Safari, we shouldn't run into structuredClone() errors due to the FileSystemDirectoryHandles.

It also seemed good to rework how we were passing mount options, but there were issues due to progress callbacks making it into postMessage payloads and causing DataCloneErrors. So I'm working on backing those changes out so this can work.

@brandonpayton
Copy link
Member

This PR should be working again in Safari and for local device sites.

@adamziel, please let me know if you have any thoughts on this. I'm planning to give this another read tomorrow or Monday and then merge if all is well.

try {
await playground?.resetVirtualOpfs();
await clearContentsFromMountDevice(mountDevice);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed this still works.

@brandonpayton
Copy link
Member

I reviewed this again and found and fixed an issue with site reset.

@brandonpayton brandonpayton merged commit ccadc7d into trunk Aug 19, 2024
5 checks passed
@brandonpayton brandonpayton deleted the worker-api-boot-method branch August 19, 2024 20:53
brandonpayton added a commit that referenced this pull request Aug 21, 2024
## Motivation for the change, related issues

The site switcher was broken by #1669. This PR is a fix for that.

Related to #1687

## Implementation details

The site that is booted depends on the redux store, so the current
loaded site now depends on redux state.

## Testing Instructions (or ideally a Blueprint)

- CI
- Use `npm run dev` and manually test switching between sites
@adamziel
Copy link
Collaborator Author

Fantastic work here @brandonpayton 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants