From af6b0e9b5c2942ad32ccb600031fb4b953011982 Mon Sep 17 00:00:00 2001 From: Domenic Denicola Date: Tue, 7 Feb 2017 18:11:49 -0500 Subject: [PATCH] Add img.decode() API for "predecoding" images This API allows the author to request that the user agent decode the image, preferably off the main thread, so that when the returned promise fulfills, the image is ready to be inserted into the document, without causing dropped frames due to decoding delay. Notable design choices made: - Rejects for non-active documents (or documents that become non-active) - Immediately fulfills for vector graphics or other formats that do not require decoding - Fulfills for animated images after all of their frames are loaded, not just the first, ensuring jank-free animation Closes #2037. This also includes the editorial change of giving a to the "update the rendering" step, and linking to it where it is mentioned. --- source | 171 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 167 insertions(+), 4 deletions(-) diff --git a/source b/source index 0a8c8c00a08..8cab36f5476 100644 --- a/source +++ b/source @@ -14953,7 +14953,7 @@ c-end = "-->" If the style sheet referenced no other resources (e.g. it was an internal style sheet given by a style element with no @import rules), then the style rules must be immediately made available to script; otherwise, the style rules must only be made available - to script once the event loop reaches its update the rendering step.

+ to script once the event loop reaches its update the rendering step.

A style sheet in the context of the Document of an HTML parser or XML parser is said to be a style sheet that is blocking scripts if the @@ -25285,6 +25285,8 @@ interface HTMLImageElement : HTMLElement { readonly attribute boolean complete; readonly attribute USVString currentSrc; [CEReactions] attribute DOMString referrerPolicy; + + Promise<void> decode(); };

Uses HTMLImageElement.
@@ -25582,6 +25584,23 @@ interface HTMLImageElement : HTMLElement { +
image . decode()
+ +
+ +

Images usually exist in some encoded form; user agents need to decode them into raw pixels + before displaying them. This process can be relatively expensive.

+ +

This method causes the user agent to decode the image in parallel, returning a + promise that fulfills when decoding is complete. The decoded image data will then be readily + available for at least one frame after the fulfillment, ensuring that attempting to display the + image will complete without decoding delay.

+ +

The promise will be rejected with an "EncodingError" + DOMException if the image cannot be decoded.

+ +
+
image = new Image( [ width [, height ] ] )
@@ -25650,6 +25669,148 @@ interface HTMLImageElement : HTMLElement {

The currentSrc IDL attribute must return the img element's current request's current URL.

+

The decode() method, when invoked, must perform + the following steps:

+ +
    + +
  1. +

    If any of the following conditions are true about this img element:

    + +
      +
    • its node document is not an active document;

    • + +
    • it has neither a src nor a srcset attribute;

    • + +
    • it has a src attribute but its value is the empty + string; or

    • + +
    • its current request's state is broken,

    • +
    + +

    then return a promise rejected with an "EncodingError" + DOMException.

    +
  2. + +
  3. Let promise be a new promise.

  4. + +
  5. +

    In parallel, wait for one of the following cases to occur, and perform the + corresponding actions:

    + +
    +
    This img element's node document stops being an active + document
    +
    This img element's current request changes or is mutated
    +
    This img element's current request's state becomes broken
    + +
    +

    Reject promise with an "EncodingError" + DOMException.

    +
    + +
    This img element becomes completely + available
    + +
    +

    Decode the image's media data entirely into its bitmap form, suitable for rapid painting to + the screen.

    + +

    If decoding does not need to be performed for this image (for example because it is a + vector graphic), resolve promise with undefined.

    + +

    If decoding fails (for example due to invalid image data), reject promise with + an "EncodingError" DOMException.

    + +

    If the decoding process completes successfully, resolve promise with + undefined.

    + +

    User agents should ensure that the decoded media data stays readily available until at + least the end of the next update the rendering step in the event + loop. This is an important part of the API contract, and should not be broken if at all + possible. (Typically, this would only be violated in low-memory situations that require + evicting decoded image data, or when the image is too large to keep in decoded form for this + period of time.)

    +
    +
    + +

    Animated images will become completely available + only after all their frames are loaded. Thus, even though an implementation could decode the + first frame before that point, the above steps will not do so, instead waiting until all frames + are available.

    +
  6. + +
  7. Return promise.

  8. +
+ + + +
+

Without the decode() method, the process of loading an + img element and then displaying it might look like the following:

+ +
const img = new Image();
+img.src = "nebula.jpg";
+img.onload = () => {
+    document.body.appendChild(img);
+};
+img.onerror = () => {
+    document.body.appendChild(new Text("Could not load the nebula :("));
+};
+
+ +

However, this can cause notable dropped frames, as the paint that occurs after inserting the + image into the DOM causes a synchronous decode on the main thread.

+ +

This can instead be rewritten using the decode() + method:

+ +
const img = new Image();
+img.src = "nebula.jpg";
+img.decode().then(() => {
+    document.body.appendChild(img);
+}).catch(() => {
+    document.body.appendChild(new Text("Could not load the nebula :("));
+});
+ +

This latter form avoids the dropped frames of the original, by allowing the user agent to + decode the image in parallel, and only inserting it into the DOM (and thus causing + it to be painted) once the decoding process is complete.

+
+ +
+

Because the decode() method attempts to ensure that the + decoded image data is available for at least one frame, it can be combined with the requestAnimationFrame() API. This means it can + be used with coding styles or frameworks that ensure that all DOM modifications are batched + together as animation frame + callbacks:

+ +
+const container = document.querySelector("#container");
+
+const { containerWidth, containerHeight } = computeDesiredSize();
+requestAnimationFrame(() => {
+ container.style.width = containerWidth;
+ container.style.height = containerHeight;
+});
+
+// ...
+
+const img = new Image();
+img.src = "supernova.jpg";
+img.decode().then(() => {
+    requestAnimationFrame(() => container.appendChild(img));
+});
+
+ +
+

A constructor is provided for creating HTMLImageElement objects (in addition to the factory methods from DOM such as createElement()):

  • Inform the user that the focus is at the location given by the intended path. User agents may wait until the next time the event loop reaches its - update the rendering step to optionally inform the user.

  • + update the rendering step to optionally inform the user.

    @@ -63396,7 +63557,7 @@ v6DVT (also check for '- -' bits in the part above) -->
  • Optionally, inform the user that the caret or selection (or both) cover the specified rectangle of the canvas. The user agent may wait until the next - time the event loop reaches its update the rendering step to + time the event loop reaches its update the rendering step to optionally inform the user.

  • @@ -88411,7 +88572,7 @@ dictionary PromiseRejectionEventInit : EventInit {
  • Microtasks: Perform a microtask checkpoint.

  • -

    Update the rendering: If this event loop is a browsing +

    Update the rendering: If this event loop is a browsing context event loop (as opposed to a worker event loop), then run the following substeps.

    @@ -120909,6 +121070,7 @@ INSERT INTERFACES HERE Silver Ghost, Silvia Pfeiffer, Šime Vidas, + Simon Fraser, Simon Montagu, Simon Sapin, Simon Spiegel, @@ -120983,6 +121145,7 @@ INSERT INTERFACES HERE Victor Costan, Vipul Snehadeep Chawathe, Vitya Muhachev, + Vlad Levin, Vladimir Katardjiev, Vladimir Vukićević, Vyacheslav Aristov,