core(user-flow): dry run flag #13837
Conversation
something seems very wrong in that 53s case if LH overhead is 50s |
| this.name = options?.name; | ||
| this._name = options?.name; | ||
| /** @type {boolean|undefined} */ | ||
| this._dryRun = options?.dryRun; |
There was a problem hiding this comment.
if this is on options why store it separately?
There was a problem hiding this comment.
this._options are the options passed to each mode runner, the type doesn't include the dryRun and name flags.
The types for this file could definitely use some cleaning up though, but I'd rather do that in a separate PR.
There was a problem hiding this comment.
The types for this file could definitely use some cleaning up though, but I'd rather do that in a separate PR.
Can you file an issue to track?
For one, Flow scripts can be arbitrarily long too. On our own flow fixtures test, dry run took the execution time from 48s to 13s (35s overhead). Since there are 4 flow steps in this test, we can estimate the LH overhead to be 8.75s per step which seems reasonable to me. |
Just curious, what's the time breakdown on that? I wonder how close dropping the audits and the waitFor conditions would get to that timing. |
|
Step breakdown normally Impact of Step breakdown w/ dry run None of these timings include any auditing.
We can try to use our own Edit: |
|
This still seems like a lot of infrastructure to support dry runs, adding a shadow version to every gather mode. It definitely makes any changes to the user flows API a lot more complicated. Was the no-gatherers-or-audits idea a no go? |
| const options = {...this._options, ...stepOptions}; | ||
|
|
||
| if (this._dryRun) { | ||
| await dryRun('timespan', options); |
There was a problem hiding this comment.
What happens to emulation when the driver disconnects? Does it persist or go away?
There was a problem hiding this comment.
https://bugs.chromium.org/p/chromium/issues/detail?id=1337089
TLDR it goes back to default, even if another session exists that is overriding the device emulation.
It helps but it's not as good as using shadow runners: A "no-audits" solution still does stuff like collect base artifacts, but the difference is small enough where I'd be comfortable doing it. |
|
Discussed, going to do the no-gatherers approach and close this. Will open an issue before closing... |
Shoutout to @BioPhoton and https://github.com/push-based/user-flow for this idea.
This allows the user to validate the user flow is performing the desired actions correctly before taking Lighthouse measurements. These speeds up the development process for user flow scripts since Lighthouse gathering can take a lot of time.
#11313