Cap images for > 2x DPR screens and use quality to reduce file size

[kurtextrem]

Describe the bug
The used srcset technique makes high-DPR devices download too big images, see the screenshots below.
At a screen width of 390px, for a 3x DPR screen (e.g. iPhone 12), it should download e.g. 750w, not 800w. For the example image, the difference is 80.8 kB -> 77.6 kB.

While 4.2 kB is not the end of the world, I do think the images are not compressed enough. Most image CDNs listed here support a quality parameter, which would drastically reduce the file size. See: https://jakearchibald.com/2021/serving-sharp-images-to-high-density-screens/

If you wonder why I say the iPhone 12 should download 750w and not 800w: https://www.andreaverlicchi.eu/capping-image-fidelity-2x-minimize-loading-time/ and a study from Cloudinary: https://observablehq.com/@eeeps/visual-acuity-and-device-pixel-ratio.
We reach a point where humans no longer see the difference, thus the additional resolution are wasted kB's. Yoav Weiss proposed to cap at 2-2.2x in Chromium: https://bugs.chromium.org/p/chromium/issues/detail?id=1125973. So for 3x DPR screens, or even 4x DPR, loading 2x is enough to have very crisp image.

To Reproduce
Steps to reproduce the behavior:

1. Paste the example HTML in the readme into e.g. a new Codepen
2. Open Dev Tools, use responsive mode and set it to iPhone 12
3. Use e.g. $0.currentSrc to find which image is currently rendered

Expected behavior

• Images are capped at 2x dimensions
• File size is reduced when applicable

Screenshots

Additional context
I've written a lengthy blog post here about this topic: https://kurtextrem.de/posts/modern-way-of-img.
<img srcset> sadly makes this a bit complicated if you compare it to the <picture> tag like used by Jake Archibald, but it is definitely possibly to control which image is loaded by using a more complex sizes attribute (which I call boolean <img> as it toggles the image around like a boolean).

[ascorbic]

Hi. Thanks for opening this. I read your blog a month or so ago. Do you think that it would be possible to automate that approach, with support for the different layouts? Right now fixed, full width and constrained each have their own rules and sizes, and any new algorithm would need that too.

[sicknarlo]

Thanks for writing that article and sharing your research. My biggest concern about implementing this (or even the other picture approach you mentioned), especially in a universal image component, is it leans more towards "telling the browser what to do" instead of "providing the browser information and letting it decide what's best". Maybe that isn't an important distinction, or maybe in some scenarios browsers are making what we think is the wrong decision, but to me the default should be using srcset how the spec is intended and saving optimizations like you suggest for special circumstances.

[benmccann]

We just launched @sveltejs/enhanced-img and landed on the same thing: https://kit.svelte.dev/docs/images#sveltejs-enhanced-img-srcset-and-sizes

[kurtextrem]

An update from my side: I spoke to Yoav Weiss from Google at PerfNow Amsterdam, and Chrome will try to get the 2x cap rolling again in 2024, probably via an attribute on the <img tag (so you can decide where to cap, e.g. to 2x, or 2.2x or 2.123x etc.). 🎊
I still see value in manually implementing capping, because Safari & Firefox have a naive algorithm of picking the next largest image... which creates an unfortunate download-waste especially on mobile (as most iPhones have a DPR of 3x). And guess who also enforces Webkit, and leads the mobile market? Right.

is it leans more towards "telling the browser what to do" instead of "providing the browser information and letting it decide what's best"

I 100% agree on that part. 'My' method, or any method that hacks around <img's limitations is definitely bossing the browser around by using (weird?) hacks to do that. However, using <picture makes all of this easy, since <picture is the API that makes you tell the browser what to do and it only picks the best fitting one with the constraints provided.

Do you think that it would be possible to automate that approach, with support for the different layouts? Right now fixed, full width and constrained each have their own rules and sizes, and any new algorithm would need that too.

I think it's tough, maybe do-able (but hacky), maybe not. I also talked to other folks about this at PerfNow, let's see what the community comes up with.

We just launched @sveltejs/enhanced-img and landed on the same thing: kit.svelte.dev/docs/images#sveltejs-enhanced-img-srcset-and-sizes

Cool, so you cap to 2x in the build (or compilation) step, right?

[benmccann]

so you cap to 2x in the build (or compilation) step, right?

Yeah. We got the idea from Next.js Image, which does the same thing

[kurtextrem]

Yeah. We got the idea from Next.js Image, which does the same thing

Nice!
I've pointed out a blind spot of using tooling to cap here: vercel/next.js#25393 (reply in thread). Not sure if this applies to Svelte (especially considering your docs; but I'm not sure if it's trivially understandable that you carefully need to craft the sizes attribute to prevent Safari & Firefox from downloading a bigger img, because of the 3x DPR)

[ascorbic]

The srcsets that we generate for constrained and fixed are capped at 2x the requested size. It's only for full width images where we offer the larger sizes, because we don't know what the size will be.

If there was a way to do this all programatically in a way that supports the different layout types then I'd definitely be interested in a contribution to support it.