Gallery: Upload images one at a time

[noisysocks]

Fixes #8935.

The problem 🤔

Note: I've tested this all using VVV, but other development and production environments would be similar.

When 5-10 large (10 MB) images are uploaded at once using the Gallery block, it's highly likely that a few The response was not a valid JSON response. errors appear on the screen.

This is because the server has responded to some of the upload requests with a 502 Bad Gateway error. The body of these responses is a nginx HTML error page, which explains the 'Invalid JSON' message.

Tailing the nginx error log (/vagrant/www/wordpress-default/log/error.log) reveals that the error occurs because its connection to php-fpm was reset:

2018/11/07 06:35:31 [error] 5676#5676: *133 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 192.168.50.1, server: local.wordpress.test, request: "POST /wp-json/wp/v2/media?_locale=user HTTP/1.1", upstream: "fastcgi://unix:/var/run/php7.2-fpm.sock:", host: "local.wordpress.test", referrer: "http://local.wordpress.test/wp-admin/post-new.php"

Tailing the system log (/var/syslog) reveals that this is because the operating system ran out of memory and so decided to kill php-fpm:

Nov  7 05:54:49 vvv kernel: [62514.273979] Killed process 7868 (php-fpm7.2) total-vm:1613272kB, anon-rss:763208kB, file-rss:8212kB

You can see from the system log that the php-fpm workers were altogether consuming about 1.7 GB of memory—easily enough to cause an OOM in this virtual machine that's configured with 2 GB of RAM:

Nov  7 05:54:49 vvv kernel: [62514.273061] [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
Nov  7 05:54:49 vvv kernel: [62514.273162] [ 7825]     0  7825   112500     2370     155        0             0 php-fpm7.2
Nov  7 05:54:49 vvv kernel: [62514.273164] [ 7868]    33  7868   403318   192855     692        0             0 php-fpm7.2
Nov  7 05:54:49 vvv kernel: [62514.273167] [ 7871]    33  7871   294628    90474     491        0             0 php-fpm7.2
Nov  7 05:54:49 vvv kernel: [62514.273169] [ 7884]    33  7884   399494    64932     667        0             0 php-fpm7.2
Nov  7 05:54:49 vvv kernel: [62514.273171] [ 7928]    33  7928   402001    90221     704        0             0 php-fpm7.2
Nov  7 05:54:49 vvv kernel: [62514.273173] [ 7930]    33  7930   244805    16504     343        0             0 php-fpm7.2

This is because WordPress generates thumbnails for each image that is uploaded. Generating thumbnails for a large image is very memory intensive. In our case, we're generating thumbnails for several large images at once.

I'm not entirely clear on why the PHP runtime itself doesn't throw an OOM exception from exceeding the memory_limit set in php.ini. My guess is that it's because image resizing happens in a seperate module running under the php-fpm process.

Proposed fix 😮

Both the Upload New Media page and the Media Library both handle uploading multiple images by uploading one image at a time. That is, we wait until the first POST request to async-upload.php finishes before starting on the second.

Borrowing this approach for Gutenberg is a nice fix for the above issue because:

• It reduces load on the server as thumbnails are generated for only one image at a time.
• It works across all of the different WordPress server environments we're likely to encounter.
• It makes the Gallery block in Gutenberg consistent with the rest of WordPress.

It's also really simple to implement thanks to async and await.

How to test 📋

1. Ensure that your PHP max upload size setting is high enough
2. Create a new post
3. Insert a Gallery block
4. Click Upload
5. Select ~10 large (10 MB) images
6. All images should eventually upload without error.
7. Check that uploading in other blocks has not regressed e.g. Image, File, Audio, Video

Future work 📆

There are some unrelated issues that should be fixed separately:

• There is no indication that an image in the Gallery block is uploading.
• There is a flash of white when an image in the Gallery block has uploaded because we swap out the blob: URL for a non-cached http:// URL.

[noisysocks]

Props to @talldan for pairing on this with me! ❤️

[talldan]

Removed the request for review since I think I've spotted an issue. Will pick up with @noisysocks tomorrow.

[noisysocks]

I pushed up some changes which:

• Fix a bug where we weren't creating all of the image blobs before starting to process image uploads.
• Rewrites the mediaUpload() tests to properly mock @wordpress/api-fetch and @wordpress/blob instead of just asserting that an error was thrown.

We should look at writing more tests for mediaUpload(). Now that its external modules are mocked properly there should be nothing stopping us from writing tests that check that mediaUpload() calls apiFetch() properly.

We should also look at splitting mediaUpload() up into some smaller functions. It is fairly large and performs a lot of array mutation which makes it really difficult to understand.

But all these problems already exist in master and I'm conscious of spending too much time here on things unrelated to the original issue 🙂

[ocean90]

Kind of a bummer that we have to revert this. Look at this 11-year-old ticket from @m. 🙂

[youknowriad]

Is this ready to be reviewed or still in progress?

[talldan]

@youknowriad Ready to review now. Just giving it a test.

[talldan]

Spotted a bug—when an image fails validation it's replaced by the next one that passes. e.g. if you try to upload images:
A B C D

and C fails validation you actually end up with the following images in the gallery:
A B D D

I'll debug it now.

[youknowriad]

LGTM 👍

This has been a long-standing issue, I'm happy to see it fixed.

[youknowriad]

(I missed the bug, let me know if you need testing after that)

[talldan]

@youknowriad - My local branch wasn't up to date 🤦‍♂️ - looks good, will merge.

[noisysocks]

Thanks @talldan and @youknowriad!

Kind of a bummer that we have to revert this. Look at this 11-year-old ticket from @m. 🙂

Yeah, wow, in five years that ticket will be able to drive.

I'm not sure that we are able to solve the general problem here which is that the server can be overloaded from requests to resize images. Server admins would have to configure nginx and php-fpm so that there's a sufficient queue for requests. WordPress doesn't have a whole lot of say in what environment it runs on.

I'd say that going forward we could look at making it so that we upload two or three images at a time. But, for now, this fixes the issue and isn't any worse than what's already in Core.

[noisysocks]

Wanted to create some issues for the future work identified here.

There is no indication that an image in the Gallery block is uploading.

This already sort of exists: #8810

There is a flash of white when an image in the Gallery block has uploaded because we swap out the blob: URL for a non-cached http:// URL.

Created: #11650