Declarative Shadow DOM

[tabatkins]

One of the promises of early Shadow DOM (and several other Parkour-descended products) was that while we would build the first version as an imperative (JS-based) API, for flexibility and composability, we'd come in later and backfill a declarative version that made it very easy to use for common cases. We haven't done this yet for Shadow DOM, and the spec is stable enough now that I think we can safely do it.

---

Rough proposal:

<div-or-something>
  <template attach-shadow shadow-mode="open | closed">
    ...shadow content...
  </template>
</div-or-something>

That is, using a <template> element with an attach-shadow attribute automatically (during parsing) attaches its contents to its parent element as a shadow root. The mode is selected by the optional shadow-mode attribute; if omitted, it defaults to "open" (or we can make it required, like the JS API does?).

---

Reasoning:

Today you can get something close to declarative with JS like:

<parent-element>
</parent-element>
<script>
"use strict";

const shadowRoot = document.currentScript.previousElementSibling.attachShadow({ mode: "open" });
shadowRoot.innerHTML = `<child-element></child-element>`;
</script>

This has several downsides:

1. Doesn't work nicely with syntax highlighting.
2. Doesn't work nicely with any tooling that wants to be able to output HTML.
3. Doesn't nest easily. (That is, having an element in the shadow contain another declarative shadow.)
4. Can't include <script> in the shadow unless you remember to do the various tricks to avoid having a literal </script> in the string.
5. Also have to be aware of which quoting character you're using, and remember to escape it when you're writing your page.
6. Including user-generated content is now more complicated; it not only needs to be escaped as safe HTML, but as safe script and string contents, too, which are decently more complex (and not, afaik, supported by the core libraries in PHP or Python).
7. Lots of non-trivial typing for something intended to be easy and widespread.
8. Inline script isn't compatible with safe CSP practice, unless you go the extra mile to add nonces (more effort! must use crypto safely!)
9. Inline script isn't run for .mhtml archives (at least in Chrome, for historical reasons).
10. Users with JS turned off won't get any page at all, for no good reason - page might have just been wanting style isolation, but is now tied to JS evaluation.
11. Inline script halts the speculative parser, I think?
12. Doesn't allow for server-side rendering (doing as much of the page as possible on the server, only using client-side JS for final fixups and event hookups, etc). The page instead has to do all this work on client-side (which is apparently fairly slow, per Declarative Shadow DOM #510 (comment))

[domenic]

I am not personally convinced of the need here (but I don't oppose it). Let me try to help with some details that could be problematic if people want to advance this.

• We don't use hyphens in built-in attribute names. So attachshadow/shadowmode instead of attach-shadow/shadow-mode
• It seems like these could be combined into a single attribute, e.g. shadow="open|closed". If shadow is not present then it's a normal template.
• We'll have to recapitulate the great defaulting wars that led to mode being mandatory in the JS API. In particular, we need to answer what <template shadow="asdf"> and <template shadow> do. Maybe the answer is nothing (as if you'd omitted the attribute), but at least for <template shadow> that seems unfortunate.
• Harder question: how does this interact with the existing processing model for templates?
  • Template contents are parsed in a completely separate document, the template contents owner document. That prevents e.g. script from running or images from downloading. It sounds like that will no longer be the case here?
  • What does templateEl.content return? Maybe it becomes an alias for templateEl.shadowRoot? Is that too weird?
  • What are the cloning and adopting steps?

[annevk]

I think I'd prefer <shadowroot mode="open|closed"> with the parser simply not treating this as a ShadowRoot insertion request if the mode attribute cannot be validated (until we can agree on a default).

This kind of feature also helps with first load where the server has done all the templating work to make rendering happen faster.

[domenic]

That would require more parser work and thus ecosystem churn though, right? On the same order of magnitude as that when we originally introduced template? Otherwise I agree it'd be nicer.

[annevk]

It should be less work now we have <template>, but probably still non-trivial. I'm not a big fan of overloading elements, that hasn't worked well historically (e.g., <object> and <embed> are still a poorly understood mess).

[annevk]

(Note that I suspect that overloading <template> would also require parser changes, due to the things you mentioned.)

[treshugart]

Hi! It's me again. Sorry. Some background:

• https://github.com/skatejs/ssr
• https://www.youtube.com/watch?v=yT-EsESAmgA
• and oh so many twitter discussions with React folks; too many to ever find and list

I've tried to keep most of the history and rationale for our thoughts in the README of the skatejs/ssr repo, but some of it might be missing. It's worth a study to at least see where we're coming from.

The thing I've found most interesting in this work is that the imperative API is exclusive in its DOM behaviour. For example, <div>test</div> means something completely different when attachShadow() is imperatively invoked. Finding a declarative equivalent that doesn't break existing assumptions is difficult. For example:

<div>
  World
  <shadow-root>
    Hello, <slot />!
  </shadow-root>
<div>

Whether or not this is used with a <template /> element or something else, there's several problems with this:

1. SEO and content order when read in engines that don't execute JS.
2. What happens to <shadow-root />? Does it get removed?
3. What happens to childNodes? Attaching a root modifies this.
4. Does appendChild(document.createElement('shadow-root')) work?

Libraries like React (and many more) have based themselves on the pillar that a core subset of DOM works as intended. If appending a shadow root mutates the host and makes childeNodes behave differently, this breaks the web. Inversely, if this is only available to the initial parse, it seems inconsistent. Meaning if <div><shadow-root /></div> is only available to the parser, but means nothing when using the imperative alternatives (i.e. appendChild(shadowRootElement)), then expectations of declarative vs imperative APIs are inconsistent.

I'm sorry if my thoughts seem a bit disjointed right now. Somebody linked this to me and it's a bit late here, but I really wanted to lay out some of my thoughts otherwise I may have forgotten about it (I do web components in my spare time).

I'm excited to see traction on this!

[matthewp]

Thanks for starting this issue @tabatkins. To add some weight that this is a need, I tried the JS approach that you mentioned some time ago and it was fairly slow: https://discourse.wicg.io/t/declarative-shadow-dom/1904/8

[tabatkins]

We don't use hyphens in built-in attribute names. So attachshadow/shadowmode instead of attach-shadow/shadow-mode

Ah yeah, right.

It seems like these could be combined into a single attribute, e.g. shadow="open|closed". If shadow is not present then it's a normal template.

I did that at first, but separated it because the mode isn't a positional attribute of attachShadow, it's just a required entry in the options dict. I thought we might add more in the future, but now that I'm thinking, I guess we can't add more required options anyway as it would break the API. So sure, we can combine them.

Maybe the answer is nothing (as if you'd omitted the attribute), but at least for <template shadow> that seems unfortunate.

Strongly agree. I think there's an obvious answer - default to open, because you're just using this as a styling boundary; nothing more special is happening at all anyway, because no JS.

Template contents are parsed in a completely separate document, the template contents owner document. That prevents e.g. script from running or images from downloading. It sounds like that will no longer be the case here?

My intention is that it'll append the template contents to the auto-created shadow root. I think scripts'll run at that point, yeah?

What does templateEl.content return? Maybe it becomes an alias for templateEl.shadowRoot? Is that too weird?

Per the above, I guess the append automatically removes them from the template's doc fragment? The answer is then ".content is empty because its contents have been transferred out". Or is there a stamping operation that doesn't remove the contents? I'm not sure of the details here. I'm flexible, whatever answer falls out is fine.

What are the cloning and adopting steps?

I think in my conception this is a parser operation more or less, so cloning/adopting just does whatever those operations do for elements that have shadow roots attached in the JS way. I'm not sure what the implications of that are, tho.

---

This kind of feature also helps with first load where the server has done all the templating work to make rendering happen faster.

Ah yes, thank you, that's another tick in the Pro column.

---

[Adding a <shadowroot> element] would require more parser work and thus ecosystem churn though, right? On the same order of magnitude as that when we originally introduced template? Otherwise I agree it'd be nicer.

I completely agree - it would definitely be nicer, but given the non-trivial parser work, I was thinking leaning on template would be better. On the other hand, I guess now that template exists widely, one can lean on that for polyfilling (doing <template><shadowroot>...) and we could add a new element. I don't have a strong opinion on this, because I understand the tradeoffs. Whatever works best for everyone.

---

SEO and content order when read in engines that don't execute JS.

There's no JS here. Tools that read the DOM would have to update to know about declarative shadow roots, sure, but that's vastly easier than supporting JS.

What happens to <shadow-root />? Does it get removed?

I'd think it would just stay there, like template does. The shadow-attaching is a parse-time operation, it's a dead element afterwards.

What happens to childNodes? Attaching a root modifies this.

What do you mean? The element's childNodes are unaffected by the presence of a shadow root. The children get distributed, which matters for CSS and some other stuff down-stream, but the DOM doesn't care at all.

Does appendChild(document.createElement('shadow-root')) work?

No, this is a parse-time operation. Post-parsing, the shadowroot element is dead, identical to a plain template. If you have JS, just use attachShadow(). ^_^

[robdodson]

The answer is then ".content is empty because its contents have been transferred out". Or is there a stamping operation that doesn't remove the contents?

I think template.content.cloneNode leaves the content in place (sorry on my phone right now so can't check).

Really excited to see this discussion happening! @kevinpschaaf @samuelli

[bedeoverend]

There's no JS here. Tools that read the DOM would have to update to know about declarative shadow roots, sure, but that's vastly easier than supporting JS.

This for me is a potential issue - if we're relying on support from SEO engines / HTML scrapers to add support, it might hurt adoption as people are after a solution with current bots / tools - particularly given the ambiguity around what bots can do already.

Is this a concern for others? If so, it might be worth exploring a declarative 'composed' DOM, rather than declarative Shadow DOM, so current bots can 'see' what the user will see. Something we're exploring over at skatejs/ssr

Apologies if this is sidetracking from the main issue, but feel its worth mentioning. Thanks for getting this discussion going!

[tabatkins]

This for me is a potential issue - if we're relying on support from SEO engines / HTML scrapers to add support, it might hurt adoption as people are after a solution with current bots / tools - particularly given the ambiguity around what bots can do already.

That's a fully general counter-argument against any addition to HTML at all, and so can't really be used as a specific objection without more details.

If so, it might be worth exploring a declarative 'composed' DOM,

That's just... DOM, right? Like, normal ordinary DOM children.

[bedeoverend]

That's a fully general counter-argument against any addition to HTML at all, and so can't really be used as a specific objection without more details.

Yeah that's fair - I'll go into specific details and whether tradeoffs are worthwhile.

That's just... DOM, right? Like, normal ordinary DOM children.

What I meant by this was declarative way to present the resultant composed tree after shadow root is attached and light DOM nodes are distributed - apologies if I'm getting the terminology wrong here.

More concretely, what I was thinking was a composed flag to tell the parser to pull a part this elements DOM into Shadow and Light DOMs.

<div composed>
  Hello <slot><strong>World</strong></slot>
</div>

which the parser would turn into:

<div>
  #shadow-root
    Hello <slot></slot>
<strong>World</strong>
</div>

This wouldn't be a 1:1 declarative version of imperative Shadow DOM API, just a way to tell the parser to construct an element's Shadow and Light trees. Straight off - and I'm sure there's a lot more I'm not considering - downsides are:

• No support for undistributed light DOM nodes
• No support for default slot content
• Certainly a confusing look at declarative Shadow DOM

I realise this may be way off base, but I just wanted to throw it out there as an idea, incase supporting current bots is worth the tradeoffs.

[treshugart]

That's just... DOM, right? Like, normal ordinary DOM children.

Yes, but this is useful. People outside of the WC community actually want this.

What we're proposing is two distinct modes:

1. Shadow DOM and CSS encapsulation via attachShadow() (the current state of things)
2. Only CSS encapsulation via the composed attribute (maybe also works via attachShadow({ composed: true }) but that can be worked out later. Declarative design first IMO).

To concretely illustrate this, React is a good example.

class Hello extends Component {
  static defaultProps = {
    name: 'World'
  }
  render () => (
    <div composed>
      <style>
        /* Works on <div composed>. */
        :host {}

        /* This works because of <slot />! */
        ::slotted(*) {}

        /* Only works for top-level span, not the slotted one. */
        span { font-weight: bold; }
      </style>
      Hello,{' '}
      {/* This span would be affected... */}
      <span>
        <slot>
          {/* ...but not this one. */}
          <span>{ this.props.name }</span>
        </slot>
      </span>!
    </div>
  )
}

The idea here is a lot like the old <style scoped /> form of encapsulation, but it differs because the full gamut of Shadow CSS works (:host, ::slotted(), etc).

The <slot /> element provides an encapsulation barrier that CSS selectors cannot cross but that's it (querySelector() still works). The assignedNodes() method does not return anything because nothing is being projected. In this mode, DOM accessors are not encapsulated. However, to turn this into full Shadow DOM, one can call attachShadow() and the composed tree is reverse engineered:

1. The host receives a shadow root.
2. childNodes of slots are attached to the host, thus are projected.
3. Any content within a slot that has the default attribute (flagging for default content) is not attached to the host, and remains as default content.
4. For named slots, this is declared as normal and is thus projected as normal.

This is literally the algorithm we're employing for declarative shadow DOM at the moment. The thing that's nice about it is that it's backward compatible and it fits very closely with the current model for shadow roots. @bedeoverend did I miss anything in that list?

EDIT

My proposal and @bedeoverend's proposal are slightly different. In his, he states that composed should reverse engineer the DOM tree whereas in mine it leaves it alone until attachShadow() is called.

[treshugart]

The "laugh" here along with the comment (not the comment alone) seems a bit condescending. We're outsiders of the W3C desperately trying to have our say in context of our extensive experience in working with these APIs and being in close working contact with other communities. Though, subtle, this downplays that and makes outsiders less motivated to participate which is bad for the specs. Can we please have a discussion here that places a bit more weight on non-W3C members' experience? Thanks!

[bedeoverend]

@bedeoverend did I miss anything in that list?

Don't think so - only thing is we're looking to handle undistributed nodes, but IMO that shouldn't be baked into platform.

FWIW I'm still unsure about whether composed should be an entirely different mode, or whether it should just trigger the HTML parser to behave differently while parsing that element. I'd be for a a different 'CSS-only' mode, but is it reaching too far right now? Perhaps there's an iterative way to get there? Keen to balance what's feasible now and what's going to garner adoption from wider web community.

[bedeoverend]

Though, subtle, this downplays that and makes outsiders less motivated to participate which is bad for the specs. Can we please have a discussion here that places a bit more weight on non-W3C members experience?

I agree with this. I appreciate it must be incredibly hard to work on specs and and the same time keep it open, but reactions which minimise external contribution only serve to ostracise the wider community. I want to try help, and I'm happy to be told my ideas aren't going to work, or I'm not communicating them well enough, but there's got to be a better way of doing that while keeping the community open.

[treshugart]

FWIW I'm still unsure about whether composed should be an entirely different mode, or whether it should just trigger the HTML parser to behave differently while parsing that element. I'd be for a a different 'CSS-only' mode, but is it reaching too far right now? Perhaps there's an iterative way to get there? Keen to balance what's feasible now and what's going to garner adoption from wider web community.

I've had discussions about this with several outside of the WC community. Since they have their own component model (whether or not they should be using Custom Elements is besides the point), they don't need the DOM encapsulation, they just want the CSS aspect. If what their internals expect changes underneath them (their declared DOM changes from what they think it looks like) then it precludes them from declarative usage. This is a barrier to entry for them because they must modify their internals.

I truly believe what I've proposed actually fits as two modes that can be composed together to achieve what you want here. It can service both use-cases while maintaining compatibility with libraries and frameworks, without requiring they update their internals.

[dominiccooney]

@annevk wrote:

I think I'd prefer <shadowroot ...

Using <template ...> would make this much easier to polyfill. For example one could write a custom element which does the parent node stuffing and the syntax would look like <template is="shadow-root" shadow="...

I guess it will be easier to add "popping the element stack and the element is a shadow-setting template" steps to the HTML parser than to separate out the context-inheriting stuff of the template to reuse it for shadowroot.

If the idea is to avoid the adoption tree walk by creating the nodes in the target document then, yeah, not complicating template is better. The inoperativeness of template parsing (specifically not running script and custom elements) would be tricky to preserve.

[hayatoito]

I tend to agree @domenic. I am not personally convinced of the need here, but I don't oppose it.

What happens to ? Does it get removed?

I'd think it would just stay there, like template does. The shadow-attaching is a parse-time operation, it's a dead element afterwards.

If it would just stay there as a dead element, the memory usage would increase, doubled or maybe more than that.

Suppose the following html, using declarative Shadow DOM via <shadowroot>,

<custom-a>
  <shadowroot>
    <slot></slot>
    <div></div>
  </shadowroot>
  <custom-b>
    <shadowroot>
      <slot></slot>
      <custom-c>
         <shadowroot>
            <slot></slot>
            <div></div>
         </shadowroot>
         <div></div>      
      </custom-c>
    </shadowroot>
  </custom-b>
</custom-b>

In this case, the composed tree which the rendering engine actually holds would be:

custom-a
  ::shadow-root
     slot
     div
  shadowroot
    slot
    div
  custom-b
    ::shadow-root
      slot
      custom-c
         ::shadow-root
            slot
            div
         shadowroot
            slot
            div
         div
    shadowroot
      slot
      custom-c
         shadowroot
            slot
            div
         div

I would prefer to remove a <shadowroot> element from the light tree after parsing and attaching it as a shadow tree, to save memory.

This can be:

custom-a
  ::shadow-root
     slot
     div
  custom-b
    ::shadow-root
      slot
      custom-c
         ::shadow-root
            slot
            div

[dominiccooney]

I would prefer to remove ` (sic) element from the light tree after parsing, to save memory.

Removing it would also make feature detection easier.

It's probably not a big deal but implementations should be able to avoid creating the template element because you won't be able to observe its brief existence (unless it is a custom element.)

[sebmarkbage]

I'll echo @treshugart's point that, for React, the shadow DOM is only overhead and complexity, when the only thing we'd really want out of this is scoped CSS. If we can't have that, we could work with a shadow DOM too. I'm sure @wycats had thoughts on this too.

I would expect many trees to result in code like:

<div>
  <template attach-shadow shadow-mode="open">
    <div>
      <template attach-shadow shadow-mode="open">
        <div>
          <template attach-shadow shadow-mode="open">
            <div>
              <template attach-shadow shadow-mode="open">
                ...
              </template>
            </div>
          </template>
        </div>
      </template>
    </div>
  </template>
</div>

Since that's semantically what a modular composed tree would be like. We'd just treat the shadow tree as the new children tree. Anything else would be an optimization (which might need to bail out).

What would be very interesting, however, is if you could target a particular ID that was defined earlier to attach the shadow lazily later in the document stream.

<section>
  <h1>Heading</h1>
  <p id="content">Loading...</p>
   ...
</section>
...
<template attachto="content">
  Hello world!
</template>

Even then, I'd prefer it if it replaced the children with the content of template, but we could live with the workaround to use shadow DOM.

[slightlyoff]

Given the extraordinary lift that was required to get <template> done (we had to get changes into XML's parsing behavior, e.g.), a new element for this seems like a non-starter.

[hayatoito]

From the perspective of implementation difficulty, whether <template newattribute> or <shadowroot>-ish new element shouldn't be a big deal, I guess, though it still needs non-trivial efforts.

From the perspective of spec, that might be a big deal.

[annevk]

@treshugart I think that by definition declarative shadow trees would have to be somewhat new as shadow trees are new. You cannot expect it to map to appendChild() just like you cannot expect that for contents of <template>.

Now, having a separate kind of feature where there is some kind of boundary that ends up applying to CSS, but doesn't involve creation shadow trees, might be worthwhile, but I don't think it's a good idea to conflate that request with the topic of this issue: declarative shadow trees. There's not even an imperative API for what you're suggesting. I have seen the request before to decouple more and I can certainly appreciate that though I haven't seen a good enough solution yet. In any event, please open a separate issue to discuss that.

@hayatoito I think we'll hit about as much complexity either way and I suspect that overloading <template> this way will make it harder to add new features to it in the future.

[domenic]

@treshugart I apologize, and have removed the offending emoji. I'll step back from this thread.

[tabatkins]

I would prefer to remove a <shadowroot> element from the light tree after parsing and attaching it as a shadow tree, to save memory.

Sure, I don't have an opinion on this. If removing it makes more sense, let's remove it. That might end up being less confusing for authors too, since the element doesn't do anything if it's sitting there.

It's probably not a big deal but implementations should be able to avoid creating the template element because you won't be able to observe its brief existence (unless it is a custom element.)

Another reasonable argument for removing it.

---

I've had discussions about this with several outside of the WC community. Since they have their own component model (whether or not they should be using Custom Elements is besides the point), they don't need the DOM encapsulation, they just want the CSS aspect. If what their internals expect changes underneath them (their declared DOM changes from what they think it looks like) then it precludes them from declarative usage. This is a barrier to entry for them because they must modify their internals.

The current concept of CSS encapsulation is built on Shadow DOM. It can't be separated from it in a reasonable way; doing so would be a completely new, totally separate feature that would need its own justification. (And I can tell you right now, it would be a hard sell to justify having two totally different ways to encapsulate CSS.) I'm not willing to try and tie declarative shadow DOM to this sort of limitation. As Anne said, this should be addressed via a separate issue.

(I also, personally, am 100% on the "stop rolling your own component models, they're not interoperable and they lock users in, just use WC" train.)

[treshugart]

Thank you, @domenic, and just to be clear - for posterity - you didn't necessarily "offend" me and it's not just you. This in isolation wouldn't be an issue. Community involvement in this working group has been pulling teeth for many, and this is one of the many forms of negativity (even if it's small) towards it. All we want is for our extensive experience to carry weight in the discussions.

I won't let this detract from the rest of the discussion. I'm over the moon this is even being discussed right now.

[hayatoito]

Sure, I don't have an opinion on this. If removing it makes more sense, let's remove it. That might end up being less confusing for authors too, since the element doesn't do anything if it's sitting there.

Regarding, template overloading (<template newattribute>) vs new element (<shadowroot>-ish),

if we can agree on "don't leave <template> element in the light tree", I prefer a new element, <shadowroot> or something, here because:

Descendant nodes of declarative shadow dom are effectively NOT inert, from user's perspective.
e.g.

  <custom-a>
    <template newattribute> (or <shadowroot>)
      <img src='...'>
      <script src='...'>
    </template>
  </custom-a>

In this case, <img> or <script> look inside of <template> element, but they would reside in the shadow tree, as the final result, after parsing html. We fetch resources for them. That sounds a big difference to me, as compared to usual <template> element's usage pattern. newattribute is too weak as a visual sign of this big different semantics.

In addition to that, as Dominic suggested, the engine (or parser) might want to optimize.

Instead of:

1. Parse a template element (and its descendants), as usual
2. Call attachShadow on the parent element of the template
3. shadowRoot.appendChild(template.content);
4. Remove template element from the light tree

We might want to optimize:

1. Parser encounters opening <shadowroot>
2. Call attachShadow on the parent element of the <shadowroot>
3. Parser continues to parse descendants of <shadowroot>, and build a shadow tree directly
4. Parser encounters a closing </shadowroot>.
5. Parser continues to parse html, and continue to build a light tree

I am not sure how this approach is feasible, but that could be an option as a starter. If we choose this, there might be no longer a good reason to use <template>.

[justinfagnani]

I can't speak to how difficult it would be to get a new element in, but @hayatoito's description of optimized <shadowroot> sounds perfectly aligned to the use-case, and not even an optimization, how how it should just work.

[justinfagnani]

@caridy something close to the snippet I posed would pretty faithfully the latest iterations of the proposal, so how would it solve your points any different than a native implementation? Aside from working with JS completely turned off, there's not much the browser can do differently from user-land code. Parser changes seemed to be completely off the table.

[caridy]

@justinfagnani no, it doesn't:

I want it to produce HTML files that are indexable by search engines first and foremost.

If it is not part of the native parsing, and everyone is doing whatever mechanism they choose to attach those shadow roots manually, how search engines can reliable determine that those templates should be analyzed and slotted? On the other hand, if is it native, spiders parsers can match the behavior of the browsers including the slotting algo.

I want my HTML to show consumable content immediately (scripts/modules are deferred because that's how we roll these days).

when is your snipped executed? at the beginning? at the end? per instance (sibling of the <template>)? it might be a good enough compromises for some, but not for all. There is a flickering effect if it is at the end, specially if you have styles that can affect the host, etc. So, it is not really solving this.

I want my HTML to be resilience even when rolling with slow or flaky networks.

Falls short for no JS, or slow networks, or any other similar scenario. In does case, the content will not be consumable.

[justinfagnani]

The markup is the same as the <template shadowroot> versions of the proposals, so the parasability is literally exactly the same. We'd need a polyfill anyway. Standardization helps, but isn't a sure thing. For instance, do we know that crawlers properly don't consider template content now? Conventions, even if not in a W3C spec can help.

when is your snipped executed?

I think sophisticated enough JS can pretty reliably emulate what the browser would do. The browser can't stream contents into the shadow roots, so it'll have to be at least chunked, if not done in one shot if the whole page is in a shadow root. Flickering can be solved as the browser would, by processing in a bottom-up up order, so that all content is still in a template until the top-level template is hydrated.

Falls short for no JS, or slow networks, or any other similar scenario. In does case, the content will not be consumable.

If the snippet is inlined with the page, it'll be fine for slow networks. For no JS situations it won't render, but the content will be there for parsing. Renderers tend to have JS though. This is the situation we'd be in if this was a standard not implemented in every browser or crawler: we'd need a JS polyfill.

[tomalec]

Flickering can be solved as the browser would, by processing in a bottom-up up order, so that all content is still in a template until the top-level template is hydrated.

The flickering I experienced with my prolyfill custom element is for the cases like:

<div>
  <template is="shadow-root">
    <link rel="stylesheet" href="make-div-green.css">
    <div>I'm green</div>
    <slot></slot>
  </template>
  <div>I'm black</div>
</div>

It flickers even if I have preloaded this stylesheet or loaded it in different shadow tree, as it still needs to load it separately here.

Then to avoid the flash of unstyled content, I need to:

1. set display: none to the host,
2. attach shadow,
3. wait for all resources to be loaded, which is tricky due to lack of link.loaded promise ,
4. remove display: none from the host.

Why it's still not enough:

• there is now the flash of no content, as I cannot block on those links,
• (needless to mention lack of stable environment or polyfill with customized build ins)

[keithjgrant]

I am disappointed this has been rejected. I hope everyone realizes this is the sort of response that this gets from the web development community:

I am sooooo glad I never really bet on the platform. React continues to innovate, while the platform continues to stagnate.
— https://twitter.com/AdamRackis/status/971758743011635202

Not that I agree with this sentiment, but… I understand where it comes from

[diervo]

As a non implementor, and someone that was in the meeting pushing for this proposal (and we do have strong use cases), I believe the push back from implementors was understandable at least with the current proposal (mostly the parsing concern and the idea that most of this could be done in user-land to a certain degree).

IMHO tweets like the one you mention don't precisely help in moving things in a good direction.

I think is up to us to give implementors more use cases and more ammunition to convince them that, this proposal, with whatever modifications, will have a indeed a huge benefit and its worth pursuing.

So we will continue pushing by giving strong reasoning, more use cases and variations on the proposal that will be easier to implement (if possible). At least thats how I believe we can move the platform forward.

[Hotell]

Very nicely said @diervo ❤️ ! It's so low to just throw sh*t out on someone and flaming with tweets like that. very sad indeed.

[treshugart]

@diervo, @tomalec, I'm interested in creating a single place for this where we can experiment on ideas, and possibly provide an official stance for an implementation. It'd also be good because those who want it know that the people who have participated in trying to push it will be operating in their best interests. This might be enough to help convince implementors over time.

Thoughts?

I am more than happy to use the skatejs org on GitHub (or even add it to the skatejs monorepo). Will invite anyone as maintainers who want to help.

[tomalec]

Great idea! I would definitely love to share my ideas and experience with others in more dedicated place.

Dedicated repo would be beneficiary, so we could iterate on proposals, provide various prolyfills, track individual problems in separate issues, etc.

Staring it at Skate org could give some recognition from the start 👍, but I think it would be nice to make it less opinionated and more framework agnostic, to reduce hype bias. So framework A fans would not disregard it just because it's framework B thing.

I think about transforming my proposal repo to a new WICG repo, given the discussion already took place at https://discourse.wicg.io/t/declarative-shadow-dom/1904 That would also give us some IP, licensing security from the start.

I just don't know if that's a correct flow from W3C perspective, as implementers already expressed lack of interest in implementing this proposal.

[annevk]

This is not the W3C, it's the WHATWG, but anyway, exploring things in a WICG repository seems reasonable. Having said that, I suspect you'll have the biggest chance of getting something to change here if you can demonstrate widespread usage of a polyfill or equivalent workaround.

[treshugart]

Could also use the webcomponents org; probably better than the skatejs org anyways. I can create a repo there if we think that's a better spot.

@annevk that's the end goal. My reasoning for proposing we put it somewhere "more official" is simply so that we can underscore the community support for it, both for people wanting to use it and, ultimately, for bringing it back to the table.

[Jxck]

I wanna see the comment for below by @rniwa about what will happen and concern to parser for understanding problem correctly.

@rniwa could you comment shortly why can't we attach shadow root to the parent and remove the element on start tag, then continue attaching descendants to the shadow root, or beside using the different name, use the mechanics of template element for parsing?

#510 (comment)

[rniwa]

We added templete element support to WebKit back in 2011 (even before the Blink fork by a Google enginner). Even as recently as a couple months ago, I was fixing security bugs related to template element in WebKit due to the way it moves the child nodes into an inert document.

The proposed semantics (of attaching a shadow root after parsing the start tag) is equally problematic because now the HTML tree builder algorithm has to be aware of shadow trees, and has to take it into account when looking at the list of open elements, etc... Because ShadowTree and Document are special root nodes of a DOM tree in WebKit (and Blink) which are directly referenced by every node in the tree, this would mean that we'd have to inspect and fix every HTML parser code to correctly update the root node (Document or ShadowRoot) referenced by a newly parsed node.

Similarly, we recently removed isindex tag which was a parser macro that expanded into a form element and an input element. This feature introduced numerous security bugs in our HTML parser due to the fact isindex itself doesn't generate an element, and removing the feature was considered a big security win. The proposed semantics of shadow element has this problematic behavior (since shadow isn't a real element).

[mildred]

If I understand well, the reason this will not be implemented is because the <template> element caused such disruption in parsers, that it can still cause bugs here and there. So a new element is not considered a sane proposal.

Then why not keep the <template> tag and introduce ways to bind the DOM of a template as a shadow DOM of another element. It can also help deduplicate shadow DOM trees too. Something like this:

<div shadowdom="hello">World</div>

<template id="hello" shadowmode="open">
Hello <slot></slot>
</template>

The shadowdom attrigute references the id of a matching template. The shadowmode attribute is inspired from https://github.com/w3c/webcomponents/blob/gh-pages/proposals/Declarative-Custom-Elements-Strawman.md

This has also the advantage of not requiring disappearing elements, or self modifying markup.

[andyearnshaw]

I agree with @mildred. The "disappearing" elements approach is counter-intuitive, no other elements in HTML work that way right now and the current proposal has lots of confusing behaviour that means a <shadowroot> element can produce either nothing (the element is removed), a HTMLShadowRootElement or a HTMLUnknownElement. There's no consideration to CSS :nth-child() or sibling combinator selectors; slow browser adoption would mean that these selectors would behave differently in some browsers. Even <script>, a single-use element that is no longer needed after it becomes connected, is not removed from the tree, and <!DOCTYPE> also leaves a traversable node in the tree.

@hayatoito

Descendant nodes of declarative shadow dom are effectively NOT inert, from user's perspective.

A template doesn't really have descendant nodes from a spec perspective, its contents aren't parsed as its children. You could quite easily suggest this can be the case for a shadow root too, but instead of hanging around until you clone them, the contents are automatically cloned into a shadow tree.

[tomalec]

"disappearing" makes it also problematic to implement a inter-operable custom element polyfill as stated whatwg/html#4566

I think the main concern and reason was the memory usage #510 (comment).

Maybe to address also #510 (comment)

The proposed semantics of shadow element has this problematic behavior (since shadow isn't a real element).

we could keep <shadowroot> element there as we keep <template>. Just make it so it does not have .children. Maybe we could keep hostElement.shadowRoot === hostElement.querySelector('shadowroot').content for open shadows, and both null for closed.

[inoas]

Thanks for feedback. We are totally aware that Declarative Shadow DOM would address some use cases, such as an alternative of scoped css.
However, our consensus at Tokyo f2f is that we are not enthusiasm to update the parser only for supporting Declarative Shadow DOM. As @tomalec mentioned in #510 (comment), supporting non-scripting environment is not sufficient motivation for updating the parser, for a while.

Unless there is another way to scope, limiting this dom feature to JS only is a step back into dark times where during Netscape 4 times if you disabled JS, CSS would stop working.

Given the adoption phase of web tech to be quite long, implementing declarative shadow at this time would boost the whole web tech stack a few years down the pipe. Why? Because together with the new 0 specifity selectors there is finally an easy way to style components and people will stop overwriting and extending huge CSS files.

The vast majority on the web are amateurs and you are denying them tools to work easy and in a clean fashion.

[drdreo]

I am currently writing my Master Thesis about this exact topic and hope we can get at least a reconsideration after finishing. I am currently looking for a similar project that first was imperative and then became declarative as some sort of comparison. Any information here is welcome.

[ionas]

Oh nice, the SSR / prerendering of (js based) web components was totally not on my radar. 👍
Ref: https://github.com/drdreo/wc-prerender

[trusktr]

@prlbr yes, but there are enough tricky questions about how this should be behave that it seems like we should see more user-space solution uptake in the wild. There's not so much that the browser can do here that can't be done with a very small script and <template>. I think if someone vended a library for this and it got uptake it might have a better chance of a comeback.

The code should be very simple, something along the lines of:

for (const template of document.querySelectorAll('template[shadowroot]')) {
  try {
    const mode = template.getAttribute('shadowroot');
    const root = template.parentElement.attachShadow({mode});
    template.remove();
    root.appendChild(document.adoptNode(template.content);
  } catch (e) {
    console.error(e);
  }
}

@caridy pretty much summed up what I also thought about @justinfagnani's idea (moreso any userland JS non-standard approach).

There are so many custom element libs out there (and there will be more and more), that we can't reliably expect them to all follow any non-standard standard. F.e. One custom element works with one SSR lib, another custom element only works with a yet a different SSR lib, etc. Things just won't work well.

As an application author, we may be able to choose a certain set of custom elements that work with a certain SSR implementation, but now we're back in the same boat as having to choose between React, Angular, Vue, Svelte. If I can't use all custom elements reliably and I have to choose only certain ones, I may as well choose any other non-custom-element framework and thus custom elements aren't winning.

An official solution will be wonderful.

[trusktr]

Is it possible to come up with some standard that would allow us to make an SSR system (server-side-rendering or statically-served-rendering) whereby absolutely any existing v1 custom element would be compatible with it (without having to modify any of the elements' implementations)? For example, a system in which we can install and use absolutely any custom element we can find on http://webcomponents.org.

@drdreo Have you found if skate-ssr (as tested in wc-prerender) has any limitations that prevent us from using certain custom elements with it (that cause the SSR system not to work as expected)? EDIT: I see you opened this issue: skatejs/skatejs#1573

Would Declarative-Custom-Elements allow us to implement such a system?

[drdreo]

@trusktr The approach of Freed would be sufficient to effectively pre-render the information hidden in a web component. Then an SSR solution needs to adapt to utilize the declarative shadow DOM. Other than that I think we are currently only limited by the spec. Declarative custom elements can handle the shadow attachment, but having the shadow API as a declarative element is, in my opinion, an important feature

[trusktr]

Seems to me even with declarative custom elements, different libs are still going to end up connecting the parts differently. How do we make there be only one way that works for everyone? (for example, we can tell all custom element authors to make equivalent camelCase props of all dash-case attributes, so that JS frameworks can always depend on passing data in by JS props instead of attributes, but we can't enforce that and we can't make that a guarantee in the current landscape).