Should createImageBitmap support vector formats?

[jakearchibald]

Step 3 of https://html.spec.whatwg.org/multipage/webappapis.html#dom-createimagebitmap for <img> specifically rejects on vector data, but it doesn't reject for blobs. There's a vague "If the image data is not in a supported file format" step for blobs, is this supposed to reject on vectors?

See https://jakearchibald.github.io/svgomg/ - as a developer, I'd like to control the rasterisation of SVG data using createImageBitmap to allow for smooth zooming and panning but lazy re-rendering at the target dimension (and cropped area).

[annevk]

See #921.

[junov]

I think #921 is a short term fix to clarify the spec, while this issue is an actual feature request.
To support the creation of ImageBitmaps from SVG media, we need to define either or both of:
a) A sizing algorithm to determine the scale of the vector graphics.
b) A version of createImageBitmap that takes size parameters (not to be confused with crop) to determine SVG rasterization scale explicitly

It should be noted that the sizing algorithm that is used when drawing SVG with 2D canvas's drawImage() cannot be use here because createImageBitmap is not tied to a canvas element that can be used as a reference for sizing.

[jakearchibald]

a) A sizing algorithm to determine the scale of the vector graphics.
b) A version of createImageBitmap that takes size parameters (not to be confused with crop) to determine SVG rasterization scale explicitly

I agree we should have this, but if an SVG has width & height attributes set then we don't need either.

[jakearchibald]

#865 proposes the size params

[junov]

In the options proposal (https://wiki.whatwg.org/wiki/ImageBitmap_Options) we had resizedWidth/resizedHeight. Perhaps we should just drop the "resized" prefix, which does not make much sense in the case of SVG. The width and height option would apply to both svg and bitmap scaling. When calling createImageBitmap with an svg source, I suppose the behavior would be to throw an exception if width/height are not specified? Also, svg images do have an intrinsic aspect ratio, so we could get away with specifying just width or height, and the other dimension could be inferred. In the case where both width and height are specified, we'd need an additional option to specify how differences in aspect ratio are handled. Something like the fill, contain and cover modes of the object-fit css property.

WDYT?

[junov]

@jakearchibald: regarding your comment about using the width & height attributes, I assuming you are referring to the image element's, computed size? I would like to avoid depending on that unless absolutely necessary. Two reasons : It is problematic for elements that are not attached to the active DOM because their style and layout is not evaluated, and it would make createImageBitmap dependent on style computation (may induce style recalc).

[jakearchibald]

It's not style dependant, it's width and height attributes on the SVG element itself. This is used to determine the <img> size if it isn't otherwise set.

[jakearchibald]

If the SVG has no defined size and no width/height set in createImageBitmap, happy with rejecting.

Also agree with width/height rather than "resize".

[junov]

So you were referring to the width and height content attributes then? That seems logical, except that the spec says that those attributes "must be omitted if the resource in question does not have both an intrinsic width and an intrinsic height".

See: https://html.spec.whatwg.org/multipage/embedded-content.html#attr-dim-width

[jakearchibald]

I'm guessing there must be something that overrides that. See http://jsbin.com/lijana/edit?html,output - the IMG here has no size specified, but it gets its dimensions from the SVG element's width and height attributes.

[jakearchibald]

https://svgwg.org/svg2-draft/coords.html#IntrinsicSizing - intrinsic sizing of SVG

[junov]

This is interesting. In your jsbin, I tried adding width="100" to the img tag, and it worked, it shrunk the car. So it seems that svg image are treated as having an intrinsic size for the purposes of this section of the spec: https://html.spec.whatwg.org/multipage/embedded-content.html#attr-dim-width

This is uncomfortably inconsistent with the behavior of CanvasRenderingContext2D.drawImage
Summoning @doomdavve and @progers

[progers]

@doomdavve, you're our only hope.

[tabatkins]

Yeah, the intrinsic sizes of an are computed from the width/height/aspectRatio attributes on its root. This gets fed into the sizing algorithms, exactly like the intrinsic sizes of raster images.

In the case that those don't exist, CSS has a well-defined algorithm that all image-y things share to determine the concrete width/height of the resource: https://drafts.csswg.org/css-images/#concrete-size-resolution. You just need to define the default object size for this case and invoke the algorithm; the output is a width and height for you to use.

From HTML:

The two attributes must be omitted if the resource in question does not have both an intrinsic width and an intrinsic height.

Wtf is this talking about. This makes absolutely no sense. I'll file a separate bug to have this line removed, it's nonsense.

[doomdavve]

I agree with all what Tab writes. And just to tie it together, https://drafts.csswg.org/css-images/#concrete-size-resolution is what CanvasRenderingContext2D.drawImage() uses to determine the width and height, with no specified size and the canvas size as the default object size.

https://html.spec.whatwg.org/multipage/scripting.html#drawing-images

I'm a bit confused about this discussion. @jakearchibald seems to be talking about the width and height attributes of the <svg>, the intrinsic size, while @junov seems to be talking about the presentation attributes of <img> which do not affect the intrinsic size. In practice, they instead map to the CSS width and height properties.

This is interesting. In your jsbin, I tried adding width="100" to the img tag, and it worked, it shrunk the car. So it seems that svg image are treated as having an intrinsic size for the purposes of this section of the spec: https://html.spec.whatwg.org/multipage/embedded-content.html#attr-dim-width

Yes, adding width="100", a presentation attribute for <img>, changes the specified width and CSS uses intrinsic aspect ratio to compute the used height.

This is uncomfortably inconsistent with the behavior of CanvasRenderingContext2D.drawImage

@junov Could you clarify the inconsistency? CanvasRenderingContext2D.drawImage() doesn't have any specified size constraints but should otherwise be quite analogous to the <img> case, AFAIK.

[junov]

Here is what I am confused about:
An <img> that refers to an svg file behaves as if the image has an intrinsic size, given by the width and height of the svg file's <svg> tag.
Use that same <img> as the image argument of a drawImage() call, and it behaves as if the image has no intrinsic size: the concrete size algorithm falls back to using the default object size, which is the size of the destination canvas + contains constraint.

[tabatkins]

Why do you say that? The behavior of drawing an <img src=foo.svg> should be the same as <img src=foo.jpg> - the intrinsic dimensions are taken from the resource. What makes you think they're different?

[junov]

https://codereview.chromium.org/1767633002/

[tabatkins]

I don't feel like digging thru a patch right now, so I'll let you summarize what it does. Are you saying that that code treats all SVG images as having no intrinsic dimensions? If so, it's buggy. width/height/aspectRatio on the root <svg> define the image's intrinsic dimensions.

[junov]

My understanding is that it considers svg images as having intrinsic aspect ratio but no intrinsic size. @doomdavve ?

[doomdavve]

It certainly considers the intrinsic size of SVG images. The thing that particular patch does is to pipe through the canvas size as default object size to use as fallback size in case the SVG image doesn't have an intrinsic size.

[junov]

Okay, I got it. Thanks for enlightening me. So SVG images may or may not have an intrinsic size. Sorry about all the confused arguments on my part. So to cycle back to the desired behavior for createImageBitmap. If the source is an HTMLImageElement, an SVGImageElement or a Blob the currently spec'ed behavior is to reject the promise if the source is a vector graphics. Instead, the behavior could be to reject the promise if the source has no intrinsic dimensions. And once we add width/height parameters to the creation options dictionary, the the behavior could be to reject the promise only if the source has no intrinsic dimensions and the width and height parameters are unspecified. Does that make more sense?

[jakearchibald]

Perfect! I'm happy with this as a first pass too, as I can easily manipulate the SVG string to add the width & height attrs.

[tabatkins]

No reason to reject images with no intrinsic dimensions. First, if they're loaded in an <img>, we can use the element's size as their dimensions if such exists. Otherwise, we just run the sizing algorithm using the canvas size (or the width/height params if specified) as the default object size.

[jakearchibald]

No reason to reject images with no intrinsic dimensions

What would the size be of an SVG blob with no width/height attrs?

const svgBlob = new Blob(['<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 10 10">…</svg>'], {
  type: 'image/svg+xml'
});

createImageBitmap(svgBlob);

[tabatkins]

Ugh, sorry, forgot the thread of the conversation. This is about ImageBitmap, so we don't have a canvas yet.

Yeah, reject things without intrinsic width and height.

[junov]

Great! I'll whip-up a PR.

[annevk]

This was fixed through #972. Thanks everyone!