Canvas Place Element

[khushalsagar]

What problem are you trying to solve?

The Web has extensive support for rendering text, both in terms of complex styling/layout and features on top of text content. For example, text selection, scrolling, translation, find-in-page, IME inputs, spellcheck, autofill, accessibility etc. This is a fundamental capability missing from canvas surfaces.

The lack of this capability forces authors to make undesirable tradeoffs: re-implement browser features when possible resulting in a sub-par experience; not use canvas even when it makes sense for their use-case.

What solutions exist today?

The current support for rendering text on 2D canvas is limited to 2 APIs: filltext() and strokeText(). These APIs limit authors to a single style, single font and single line of text. There's no support for multi-line text, authors need to manage their own layout. They also need to re-implement all browser features on top of these APIs.

WebGL provides no first class support for text.

How would you solve it?

The ability to render HTML elements on Canvas surfaces. Adding this capability enables Canvas surfaces to benefit from all of the styling, layout and behaviors of HTML, including interactive elements and built-in accessibility.

Anything else?

This feature is being incubated in WICG following the WHATWG standards process.

Explainer: https://github.com/WICG/canvas-place-element/blob/main/README.md
Ongoing prototyping in Chromium
Contributor: Google, Igalia
Workstream: HTML

[khushalsagar]

We'd like to move to Stage 1, links to relevant documents are in the comment above.

[smaug----]

I think I might want to see the explainer go through the known issues in some more detail before moving this to stage 1. Especially hit testing can be rather problematic (some issues mentioned in WICG/canvas-place-element#9) and it is unclear how various APIs dealing with coordinates would work.
And also things like IME popups - since the element's layout depends on the canvas itself, it is unclear how IME related popups can position themself correctly, if the visual representation on the canvas is in totally different place.

The problem space itself is interesting. Reminds me of mozSetImageElement or drawWindow.

[annevk]

cc @whatwg/canvas

[khushalsagar]

Thanks for the feedback @smaug----. I’ve replied to the hit testing and IME popup points in WICG/canvas-place-element#9. I’m curious to hear if the experience with mozSetImageElement gives us some ideas here.

That said, if I understand the stages correctly, the fact that we’re aligned that the WHATWG should explore this problem space implies we can proceed to stage 1. The questions you’re raising is exactly the kind of iteration/engagement we want to move this feature to. Stage 2 (iteration) is what would need all these issues to be resolved.

[smaug----]

I was referring to "A comprehensive explainer " as requirement for stage 1. A bit nitpicking, I know, but there were quite a few unknowns still.