-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Fix perf issue with command log #21007
fix: Fix perf issue with command log #21007
Conversation
Thanks for taking the time to open a PR!
|
31eb4f5
to
099d51d
Compare
@andrewmtam We appreciate the work put into this PR, however exposing a configuration option that would limit the number of commands shown is not a feature that we want to add. We would prefer a solution that addresses the performance issues without degrading the experience of using the entirety of the Cypress Test Runner at the time it is run. Not being able to see x commands degrades the experience of the runner significantly as information is essentially lost from that run. Ideally a rendering solution that would show the relevant commands and relevant data upon scroll. So far our investigations into a rendering solution like this that would improve performance have not given us any clear solutions. |
Thanks @jennifer-shehane , appreciate the feedback! I've got two follow up questions if you've got a moment:
out of a configuration option and into an environment variable, but seeing that we already have an environment variable that hides the command log altogether, I just wanted to double check whether or not doing the same thing for limiting the command log is also off the table. My main push for this point is that, if consumers set
|
Ooo one more idea came to mind:
Clicking "Show More" each time could increase the amount shown by 25. So on first click, 50 commands would show, then 75, 100, etc etc. But if the user really does want to see everything, they just click "Show All", and it'll behave as-if th command log was never limited |
Hey @jennifer-shehane , I wanted to give one more go at this and put up a PR here: This does not implement the ideal solution you hinted at, but it does solve this problem:
by providing a way to surface the entire command log list and allow users to click through snapshots from the whole test run. Curious to hear your thoughts on this approach if you've got a couple minutes! |
.each()
run slower in Electron duringcypress run
in 3.8.0 #5987User facing changelog
Added a new configuration option,
numCommandsToShow
, which can be used to yield performance improvements for long running tests.This will limit the number to commands that show in the Command Log (the left sidebar) of a cypress test, instead of allowing it to grow indefinitely.
Additional details
We already have a parameter,
CYPRESS_NO_COMMAND_LOG
, which hides the Command Log altogether. However, this is not sufficient for both debugging and performance.numCommandsToShow
seeks to provide a solution that satisfies both performance and debugging concerns.While this does restrict the user from clicking on snapshots past the specified
numCommandsToShow
value (and only if they opted into this), I don't think this is typically needed (when debugging, we only need the last couple of commands around when the test failed)Related docs PR:
cypress-io/cypress-documentation#4431
How has the user experience changed?
I believe this simple change / configuration can yield huge huge performance gains for users, with little to no risk of regression at all.
I ran a cypress test w/o this change, and repeated the same series of steps in a test 1000 times (I just took screenshots because the video was too big, and I also didn't bother waiting to get to 1000).
The test was to:
From the screenshots, we can see that this step started off taking 400ms, and very quickly we found ourselves at the 1200ms mark, a 3x increment from when the test started, and after only 100 iterations.
Before applying this PR
After applying this PR (using a
numCommandsToShow
of 25)Next, I applied this PR, and adjusted the new configuration option, numCommandsToShow, to 25.
For the duration of the test, the duration of the same sequence of actions did not linearly increase with the number of commands getting run. Also, note that at the 100 iteration mark, this test took /50%/ of the time as the previous test, 42s vs 87s.
PR Tasks
cypress-documentation
?type definitions
?cypress.schema.json
?