Shadow DOM support

[letsgotomars]

🚀 Feature Proposal

Support for Shadow DOM in the IDE

Motivation

Allow Selenium IDE to work with modern websites that use Shadow DOM. Currently it fails to select targets in shadow dom

[letsgotomars]

Even a cludgey plugin or non-elegant workaround would be very much welcome.

[letsgotomars]

Just pinging the active maintainers @corevo and @tourdedave to make sure this doesn't fall off the radar. Our team really would like to use the IDE but all our applications use ShadowDOM so in the IDE's current state this is not possible for us. Our team might potentially be able to lend some support in this area depending on the scope of the work to make this possible.

[corevo]

Sorry for not responding earlier, our current stance is that we will support whatever the webdriver protocol will support.

Currently webdriver have yet to standardize shadow DOM, once they do we will implement that in the IDE.

As a general rule we will not stray away from webdriver.

[letsgotomars]

No worries thanks for the udpate @corevo. It's unfortunate there is no webdriver support for this awesome modern web feature yet. I hope the standards community implements some sort of shadow DOM support in the webdriver spec soon because the lack of automated testing will greatly discourage shadow DOM's adoption and use.

The W3C webdrvier issue tracking shadow dom support that I could find is here: w3c/webdriver#350.
(Looks like there might be some promising recent progress on this front with w3c/webdriver#1320)

In case other folks see this issue who are struggling with this same problem of testing their shadow DOM pages here are some other IDEs/tools that are working & tracking this issue:

• Katalon Studio - manual workaround

• TestCafe Studio - shadow dom support issue

• Cypress.io - shadow dom support issue

[letsgotomars]

Until there is official webdriver standards support for shadow DOM I might have some of our devs look into forking the Selenium IDE with a stopgap workaround solution (perhaps using something like this or this).

Since the contributing guidance for the IDE seems to be mostly TBD at this time, @corevo is there any quick info you could give us on how to get started? Any advice is not expected but would be greatly appreciated

[corevo]

I would suggest working on a plugin for the time being, so people who still want to use the official IDE and have shadow DOM support can do so. Cheers, Tomer

…

On 7 Jun 2019, 23:45 +0200, letsgotomars ***@***.***>, wrote: Until there is official webdriver standards support for shadow DOM I might have some of our devs look into forking the Selenium IDE with a stopgap workaround solution perhaps using something like this. Since the contributing guidance for the IDE seems to be mostly TBD at this time, @corevo is there any quick info you could give us on how to get started? Any advice would be greatly appreciated — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[ghost]

@corevo can you please point out where I can find documentation for plugin development? Basic example of custom element finder would be highly appreciated. Also, is it possible to use plugins with cli tool, to use them on CI?

[ghost]

@corevo @tourdedave anyone?

[tourdedave]

@dcbrwn https://selenium.dev/selenium-ide/docs/en/plugins/plugins-getting-started

[ghost]

@tourdedave https://selenium.dev/selenium-ide/docs/en/plugins/plugins-getting-started#locators looks like I need to implement custom locator and it seems undocumented or I'm missing something. Docs are a bit outdated. I assume xpath and css locators are implemented as plugins internally? Could you give a hint how to find them?

[corevo]

Implementing a locator isn't currently feasible as a plugin, not to discourage but a different approach I would investigate is adding a command to switch into a shadow DOM context, rather than adding it on top of locators.

[ghost]

@corevo in our app we use Polymer and lately LitElement. Every component there has shadow dom. So it means that we'll need to switch between shadow roots dozens of times in each test-case. This is possible but very inconvenient and renders tests unreadable. Currently we use custom locator strategy in wdio and select elements by special attributes, not xpath nor css. Having something similar in SIDE would be of great help! If there is any design for custom locators I guess I can help implementing it.

[corevo]

There is a problem releasing that, since the recorder by design is synchronous and the plugins are asynchronous.

Currently to add a custom locator you would have to build the IDE yourself.

[ghost]

@corevo so asynchronous recorder is already implemented but not merged? If there is a PR can you give a link please? If I can help I will, otherwise I'll just try to implement the plugin for that branch and wait for it to be merged.

[corevo]

No, asynchronous recorder isn't possible atm, but a synchronous plugin system will be possible in the electron version of the IDE.

So not soon.

[corevo]

By building the IDE I meant, build one with the shadow DOM changes in it, so that they can be synchronous.

[letsgotomars]

It appears that support has finally landed in the W3C Editor's Daft spec 🙌. @corevo does that mean the Selenium IDE will be able to natively support Shadow DOM once browsers have implemented the spec? If so would we be able to see this feature in the near future and what would the path forward look like?

As a maintainer myself I understand that the Selenium IDE team has to make many complex and difficult decisions about what features to prioritize in addition to significant refactoring efforts and bugfixes so I will simply state my case for why I think this is a compelling feature that deserves attention and prioritization:

1. This issue has the most ❤️s and 3rd most 👍s on this repository
2. Due to popular demand many test automation tools such as Microsoft's new Playwright and the aforementioned tools now have built-in Shadow DOM support. There is obviously growing demand for this capability in the test automation community and I would hate to see the Selenium IDE left behind.
3. Recently the adoption and usage of Shadow DOM and been rapidly accelerating and will continue to rise as the Shadow DOM spec, developer community, and supported software ecosystems continue to advance and improve. For instance adoption has grown by over %500 just since this issue was first opened. From https://www.chromestatus.com/metrics/feature/timeline/popularity/804

[lgext06]

Hi Folks, @corevo @letsgotomars @AutomatedTester

A year and a half later, I wonder what is the current status/short term plan of the support of Shadow DOMs support in Selenium IDE.

It looks like W3C has defined some specification around this (https://w3c.github.io/webdriver/#shadow-root), still as a draft (I do not know exactly what this means btw ...).

We can cope with it with various workarounds when coding tests, but recording and replaying scripts in the IDE is not possible.

What are the next steps then ?
Can we expect an update in Selenium IDE soon?

The shadow DOM appears to be more and more used in Web applications:

[controllerface]

Hi all,

I am interjecting slightly here (as I'm not a contributor nor a user of selenium) but I work on a DAST tool that has support for the "selenese" commands for scripting, and the lack of support for this was an interesting problem for us for a while.

We waited a bit to see if support would materialize so we could direct customers to use the "official" syntax, but since it never happened we implemented support as a custom pseudo-element ::shadow-root, so for example, css=html > body > my-custom-element::shadow-root div would let you access a div inside the shadow root of an element.

I realize this is not actual syntax that would work for querySelector() which is kind of the intent of the css= selector type, but this felt like the most correct way to accomplish this.

I realize that the selenium IDE team may not be able to just adopt this convention, and I'm almost certain browsers never will, but this approach has been saving our bacon a few years now and works extremely well all things considered, since you just have to append the pseudo-element to the proper elements in the selector.

Just tossing it out there, this is not an easy issue to resolve in a client side plugin, so you all have my sympathies. Best wishes!

[IMCubator-CI]

May the Force of the May the 4th lets the Shadow DOM Support roll to our loved Selenium IDE.