-
Notifications
You must be signed in to change notification settings - Fork 269
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add import and export logic #110
Add import and export logic #110
Conversation
As a next step, I'd like to incorporate the database changes into the import / export functionality. The default WordPress export works; however, I'm running into an issue with the WordPress Importer, namely that it doesn't install. I'm not sure what the best approach here is. Maybe the WordPress Importer should be installed by default with Playground. I haven't looked into this in depth yet, but I imagine I'd need to hook into that when doing the import functionality on my end, at least code-wise even if the UI doesn't work. I could also take a look at this myself as part of this issue and pull request if it would be helpful, though would appreciate any pointers to set me off in the right direction 🙏 wordpress-importer.mp4 |
It was pointed out to me that it's possible to install the WordPress Importer and other plugins using a query string as follows: With that in mind, will continue working on this issue and incorporate the database changes as well. |
This is awesome @artemiomorales, thank you so much for looking into this! I'll leave some more notes tomorrow, but for now:
As a rule of thumb, tiny libraries are mostly fine. JsZip is 100kb minified. It may not be much compared to 5MB of WebAssembly code in php.wasm file, but I would still like to add as little on top of that as possible. If it can be replaced by PHP's ZipArchive then great. If not, let's explore loading it async with |
I took it for a spin @artemiomorales and it's lovely! I tried commenting out the
I think your intuition behind the importer plugin is right – it should take care of all the necessary adjustments 🤞
Yeah it would make sense to bundle it with the default WordPress installation. It would mean adjusting the WordPress build script and committing new bundles to this repo. As a side note, eventually it would be nice if GitHub handled the build pipeline automatically. Until then, we'll need to periodically re-run
Take a look at wp-macros.ts where Playground handles logging the user in, installing the plugins etc. I imagine the importer workflow could leverage the same idea of basically controlling the website like a Puppeteer test would. All in all, splendid work so far! |
21d299d
to
7bc02d5
Compare
@adamziel Here's an updated screen capture showing the current state of this feature. We now have exporting and importing of the database alongside the file system changes, and all of the ZIP processing is now happening in the worker thread on the PHP side. import-and-export-update.mp4Key considerations:
Please let me know your thoughts 🙏 If we can think of a targeted, primary use case of this feature, maybe that can help us nail down the user experience. I'll think on this more. |
This is awesome @artemiomorales! Thank you so much for working on this! Some notes:
I haven't given the UX part of it much thought yet. Maybe it would make sense to always ship that plugin preinstalled, or at least install it when the import/export feature is enabled? Just like you get logged in as an admin when you request any
There's a larger question about the project structure here. Playground as an isolated site on https://wasm.wordpress.net/wordpress.html has limited usefulness. Its real power is the ability to embed it in other apps. Some importing/exporting use-cases I came across are:
None of these would involve an upload form directly in the playground – they all need a dedicated user interface. Consider the "settings button" in the "browser window" on https://developer.wordpress.org/playground/. It's not implemented in this repository but is a part of the Wasm Demo Gutenberg block which lives in a dedicated wp.org theme. All this tells me that an import/export needs to be exposed to developers via some form of API, be it a The playground itself only needs a basic UI to demo the feature. There could be a set of export / import buttons somewhere in the browser UI and all they would do is trigger a file download and bring up a file upload form. They could be visible by default unless disabled with What do you think? I'm happy to be wrong about any of these.
WordPress assets are often sourced a version string like
That sounds awesome! See the existing tests in |
CC @bengreeley as this is highly relevant for live previews in the plugins and themes directories on WordPress.org |
Great progress, @artemiomorales! For the caching delay, global styles are currently set to a 1-minute transient. The final fix to core is still being deliberated, but this PR shows promise if you're looking for a patch before 6.2 drops: WordPress/wordpress-develop#3712. |
As follow-up:
Note that WordPress/wordpress-develop#3712 has been merged: Use a non-persistent object cache instead of transient in |
@adamziel I took a look at the new CLI functionality, but I think it makes more sense to install the WordPress Importer via the Dockerfile. The reason for this is that, if we use WP-CLI to manage installation and activation of the WordPress Importer, the way that WP-CLI parses the wp-config file conflicts with the removal of whitespace from the PHP files. Since the stripped version is what exists in the WordPress bundle, opting to get the importer with WP-CLI during the build process seems to be the most straightforward way of doing things. With that in mind, I moved the stripping of the whitespace until after WordPress Importer is installed, making sure to avoid stripping the importer files, as modifying them prevents the plugin from working properly. I also went ahead and rebuilt all of the WordPress bundles, and the export / import seems to be working in all cases. One issue though is that full site editing in 6.0 and 6.1 appears to have broken during the build process. I'm planning to look further into what's causing that issue. Another limitation is that we can't go between different versions of WordPress — for example, if you export a WordPress 5.9 instance, you can't import that into a 6.0 instance. Exporting also only works with PHP versions 7.4 and up, and I'm planning to look into ensuring compatibility with other versions of PHP as well. I'm considering how far to take this and whether we should cover going between different versions of WordPress right now. I think it may be worth it to document that as a known limitation, and just fix full site editing, PHP compatibility, and the CSS cache issue, along with adding the UI, to ship this and get more eyeballs on it. Would be happy to hear your thoughts! |
That sounds great @artemiomorales! 👍
Could that be related to this? wordpress-playground/src/wordpress-playground/wordpress/Dockerfile Lines 45 to 74 in 9c7d797
That's a good point! Is this how WordPress works in general? Or is that specific to Playground? If it's the latter – what makes Playground different?
I wonder if that's because of the following part of the PHP build process: wordpress-playground/src/php-wasm/wasm/Dockerfile Lines 491 to 495 in 9c7d797
Does the
A known limitation sounds great 👍 |
I just checked the readdir function on PHP <= 7.3 and it works correctly: console.log(
new TextDecoder().decode(this.run({
code: `<?php
$handle = opendir('/');
while (false !== ($entry = readdir($handle))) { echo "$entry\n"; }`,
}).body)
); The problem you're running into @artemiomorales must be caused by something else then. |
d774211
to
4ef16fd
Compare
For anyone following along, @adamziel and I took a look at this together — it was related to the build process deleting a file needed for full site editing to work. That's now been fixed on trunk.
Not sure! We've determined though that this can be a known limitation for now. I've made a note to create an issue for this.
This is something I'll also look into later — will make an issue for it. |
I just pushed up the UI for this migration logic. Here's a demo. Would be happy to hear feedback 🙏 Note: The cache issue still exists. I tried patching it using the fix on WordPress trunk, but it seems getting that bug cleared may be beyond the scope of this feature; likely we can ship with the cache issue as a known limitation until the next WP version is released. cc @adamziel migration-logic-ui-1080.mp4 |
This is lovely @artemiomorales! A few notes from me:
Otherwise it looks great and I'm excited to get it in! CC @fabiankaegy |
Ah, and could this PR not affect the rebuilt assets in |
…cludes/' and 'wp-content/database/' directories
Added back the empty.html file that is needed for full site editing to work. Removed an erroneous command in the package.json that was being used to build WordPress bundles, as our intent is for specific versions of WordPress to be specified when building WP bundles.
Export functionality was not working due to PHP not being able to read a file after it had been written. This was due to insufficient file permissions; that has now been fixed. I also added the PHP version to the name of the export file.
In order to test the behavior of the migration feature's PHP code, I moved that code to separate files so that it can be run by Jest. This required modifying the generateZipFile() function a bit to accept parameters.
Moved part of the import logic to separate file, modified it to use parameters, and added many constants in the migration test file to prevent code duplication.
It turns out the code was not importing files if their parent directories didn't exist. That has now been fixed, and the code has also been moved to a separate file to allow for testing.
19bfb46
to
0badcd2
Compare
@adamziel Tests have been added; I even caught a bug while adding them 🙌 Let me know what you think. If all looks good, I think this feature is ready to ship. |
) as HTMLButtonElement; | ||
const importCloseModalButton = document.querySelector( | ||
'#import-close-modal--btn' | ||
) as HTMLButtonElement; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's about time this entire file gets migrated to (p)react – that's definitely out of scope of this PR, though
Thank you @artemiomorales ! I left a few nitpicks but it looks good overall. I'd like to merge as soon as these are addressed. For posterity, here's a few tasks I'd like to look into next:
All of these are out of scope for this PR but would greatly benefit the repository. |
Co-authored-by: Adam Zielinski <adam@adamziel.com>
@adamziel Great, this PR has been merged and I created issues for all of the above. |
@artemiomorales This is great! As a side note, please use the squash&merge feature when merging future PRs to avoid merging the entire PR history to trunk. |
This pull request contains functionality for exporting and importing changes from a Playground instance. It renders some debug input fields to allow you to play with it.
This PR addresses issue #88. Note: Currently, this only exports and imports the filesystem changes, not the database.
How to Test
npm ci
import-and-export.mp4
Notes
This currently makes use of the JSZip and FileSaver.js. How do we feel about adding those dependencies?