Memory leak in long running single test - crash #4164

Konstruktour · 2019-05-08T19:35:35Z

Current behavior:

Currently if a test runs the same actions repetitively over a long period of time, the memory of the cypress process increases until it crashes. (headless mode, video off, numTestsKeptInMemory 0)
The sample to reproduce runs 15 minutes increasing the memory ~300MB. (the longer the run the more memory is used..)

Desired behavior:

Performance tests should be possible with cypress, without increasing memory and crashing.

Steps to reproduce: (app code and test code)

Config cypress.json

{
  "baseUrl": "http://localhost:8080/",
  "defaultCommandTimeout": 60000,
  "viewportWidth": 1280,
  "viewportHeight": 720,
  "numTestsKeptInMemory": 0,
  "video": false
}

Cypress test

describe('memory dummy test', () => {
    const endTime = Date.now() + 60000 * 15;

    before('visit', () => {
        cy.visit('/memory-test.html');
    });

    it('should test', () => {
        const test = () => {
            cy.get('#testinput').clear().type('hello world');
            cy.wait(2000);
            cy.then(() => {
                if (Date.now() < endTime) test();
            });
        };
        test();
    });
});

Simple static html site

<!DOCTYPE html>
<html lang="en">
<head>
</head>
<body>
	<input id="testinput">
</body>
</html>

Versions

Cypress 3.2.0 / 3.5.0 / 3.6.0
Windows 10

scharf · 2019-06-16T16:38:12Z

see #1906 (comment)

brian-mann · 2019-06-17T12:47:20Z

I actually believe this issue will be fixed by this PR which will be merged in the next patch release: #4406

brian-mann · 2019-06-17T12:48:06Z

It was an oversight / bug that we were still taking snapshots when a single test even though numSnapshotsKeptInMemory was 0 - so it's expected that the memory would grow larger after each command runs - until the next test runs.

brian-mann · 2019-06-17T12:48:32Z

Closing this issue as duplicate because I'm very confident this is describing the same behavior as #4104

cypress-bot · 2019-06-27T23:31:37Z

Released in 3.3.2.

Konstruktour · 2019-11-04T14:09:43Z

@brian-mann / @jennifer-shehane: Retested it with version 3.6.0... still the same issue! :(

Konstruktour · 2019-11-08T12:00:49Z

@brian-mann / @jennifer-shehane: can this issue be reopened?

jennifer-shehane · 2020-01-03T07:47:11Z

This issue will be closed to further comment as the exact issue here was resolved and tested.

If you're experiencing a bug similar to this in Cypress, please open a new issue with a fully reproducible example that we can run. There may be a specific edge case with the issue that we need more detail to fix.

jennifer-shehane · 2020-02-05T16:02:47Z

I ran the provided code and was unable to reproduce the crashing described in the original issue. The spec ran for 15 minutes and finished to completion in Cypress 3.8.3 in Electron 78.

*Couldn't fit log in one screenshot

There will be a new PR in 4.0 that will log process usage that would be helpful to see what is going on here memory wise #6171. How did you assess this is consuming more memory over time?

Konstruktour · 2020-02-05T17:04:08Z

thx for getting a look again at that issue...
the crashing only occurs if the system gets out of memory.. the consumption increases over time, 15 min will be too less see the crash ...
nevertheless you can track the memory consumption in the task manager..
eg the test started with ~100mb memory usage:

after 45min it increased to ~360mb ... and rising...

Konstruktour · 2020-02-13T10:26:45Z

@jennifer-shehane could u recreate the issue? any new findings?

Oliboy50 · 2020-07-18T14:08:15Z

@jennifer-shehane if you want another easily reproductible example
=> I guess I'm experiencing something similar in this project on cypress 4.10.0

it runs a single test which is pretty fast at the beginning and starts to be really slow after ~50 seconds of execution

I tried to use "numTestsKeptInMemory": 0 cypress configuration but it didn't help at all 🤷

if you want to try it out, please install node v12.x and run the following commands:

git clone https://github.com/Oliboy50/coinche.git
cd coinche

cd client
npm install
cd ..

cd server
npm install
cd ..

cd e2e
npm install
npm run dev

Cypress GUI will pop up and you'll be able to run the test I'm talking about

hope it will help (I'd love to have a faster test, one which does not make my computer burn ❤️)

jennifer-shehane · 2020-10-08T02:49:38Z

We're investigating some performance issues related to the rendering of the Command Log such as #6783 where I think the solution to that may also solve this issue. Will have to investigate once there is a proposed PR there.

pardamike · 2020-10-09T21:47:38Z

Our team has worked around this issue by simply using:

launchOptions.args.push('--max_old_space_size=1500');
launchOptions.args.push('--disable-dev-shm-usage');

inside of our plugins/index.js

Note that we are on an older version of Cypress and older image, and we are running this on Gitlab CI:
package.json: "cypress": "^4.12.1"
.gitlab-ci.yml (image): name: cypress/included:4.9.0

Hopefully this can help someone else 👍

dmolin · 2023-03-21T08:40:26Z

Any update on this? this is a huge road blocker in our project. We've long running tests that inevitably end up with the browser crashing bc of out of memory errors. Is there any actual activity/research currently done on this issue?

krisodb · 2023-03-21T09:41:01Z

Check #25557

I'v also added env var CYPRESS_NO_COMMAND_LOG=1 on our pipeline.
Not much added value having the command log for remote runs.

I've added some memory graphs on twitter: https://twitter.com/op_kris/status/1636675020180058112?s=46&t=DPClh6K4rCQElpP_CUCryg showing the difference between the usage of the command log or not.

CloneOfAlex · 2023-07-20T00:43:49Z

any updates ??

Error code: Out of Memory

nagash77 · 2023-07-25T12:40:57Z

@CloneOfAlex In order to give our engineers the best chance at recreating your problem, please create a reproducible example using a fork of Cypress Test Tiny. This gives us the best chance of seeing exactly what you are seeing and being able to investigate effectively.

StefanTegeltija · 2024-01-04T15:26:41Z

Could you please inform me if you haven't managed to reproduce this issue from 2019 so I can make reproducible example for you?

scharf · 2024-01-04T18:01:16Z

I have seen out or memory errors this recently

CloneOfAlex · 2024-01-04T19:08:30Z

@CloneOfAlex In order to give our engineers the best chance at recreating your problem, please create a reproducible example using a fork of Cypress Test Tiny. This gives us the best chance of seeing exactly what you are seeing and being able to investigate effectively.

Sadly I cant share our code

BUT -- on 16GB laptop when running with local runner , Cypress 13.3.3 / Node.js 20.10.0 - often runner crashes with out of memory error .. // hence forced to run only few scenarios at a time ..

Thanks!

StefanTegeltija · 2024-01-05T09:14:35Z

Hi @nagash77 , I have created reproducible example of this issue: https://github.com/StefanTegeltija/memory-leak-in-long-running-single-test, sorry I haven't used for of Cypres Test Tiny but it should not be hard to run my example.
In order to run tests you just need to:

Download this repo
Run npm install
Run npm run run:dev
You will probably see issue in every run :)
Please let me know is you need anything else :)
Chrome version: Version 120.0.6099.71 (Official Build) (x86_64)

LukasLewandowski · 2024-03-19T14:28:34Z

Any updates for memory problem?

CloneOfAlex · 2024-03-19T14:30:20Z

Hi @nagash77 , I have created reproducible example of this issue: https://github.com/StefanTegeltija/memory-leak-in-long-running-single-test, sorry I haven't used for of Cypres Test Tiny but it should not be hard to run my example. In order to run tests you just need to:

Download this repo

Run npm install

Run npm run run:dev
You will probably see issue in every run :)
Please let me know is you need anything else :)
Chrome version: Version 120.0.6099.71 (Official Build) (x86_64)

What Node.js version you have installed ??

do node -v and let us know

Also -- you are using OLD Cypress version .. : (

Why dont you install:

Node.js 20.11.0 & Lates version of Cypress 13.7.0

StefanTegeltija · 2024-03-19T14:31:45Z

v16.18.1

CloneOfAlex · 2024-03-19T14:32:18Z

v16.18.1

You are using obsolete Node and Cypress versions ..

Install following and try again:
Node.js 20.11.0 & Latest version of Cypress 13.7.0

CloneOfAlex · 2024-03-19T14:34:51Z

after installing latest Node and Cypress

also run

npm outdated

next mitigate outdated packages

run

npm install
npm audit fix
npm outdated

StefanTegeltija · 2024-03-19T14:41:38Z

So you managed to fix this issue by these suggestions using my repo?

StefanTegeltija · 2024-03-19T14:44:15Z

BTW the newest version of cypress cannot be downloaded for some reason. npm WARN deprecated @babel/plugin-proposal-class-properties@7.18.6: This proposal has been merged to the ECMAScript standard and thus this plugin is no longer maintained. Please use @babel/plugin-transform-class-properties instead.

npm WARN deprecated @babel/plugin-proposal-object-rest-spread@7.20.7: This proposal has been merged to the ECMAScript standard and thus this plugin is no longer maintained. Please use @babel/plugin-transform-object-rest-spread instead.
npm WARN deprecated intl-messageformat-parser@1.8.1: We've written a new parser that's 6x faster and is backwards compatible. Please use @formatjs/icu-messageformat-parser
npm WARN deprecated gherkin@5.1.0: This package is now published under @cucumber/gherkin
npm WARN deprecated cucumber-expressions@5.0.18: This package is now published under @cucumber/cucumber-expressions
npm WARN deprecated cucumber-expressions@6.6.2: This package is now published under @cucumber/cucumber-expressions
npm WARN deprecated cucumber@4.2.1: Cucumber is publishing new releases under @cucumber/cucumber
npm WARN deprecated puppeteer@9.1.1: < 21.5.0 is no longer supported
npm WARN deprecated core-js@2.6.12: core-js@<3.23.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Some versions have web compatibility issues. Please, upgrade your dependencies to the actual version of core-js.
npm ERR! code 1
npm ERR! path /Users/stefantegeltija/Desktop/projects/cs-qa-automation/node_modules/cypress
npm ERR! command failed
npm ERR! command sh -c node index.js --exec install
npm ERR! Installing Cypress (version: 13.7.0)
npm ERR! 
npm ERR! [STARTED] Task without title.
npm ERR! The Cypress App could not be downloaded.
npm ERR! 
npm ERR! Does your workplace require a proxy to be used to access the Internet? If so, you must configure the HTTP_PROXY environment variable before downloading Cypress. Read more: https://on.cypress.io/proxy-configuration
npm ERR! 
npm ERR! Otherwise, please check network connectivity and try again:
npm ERR! 
npm ERR! ----------
npm ERR! 
npm ERR! URL: https://download.cypress.io/desktop/13.7.0?platform=darwin&arch=x64
npm ERR! Error: Failed downloading the Cypress binary.
npm ERR! Response code: 502
npm ERR! Response message: Bad Gateway
npm ERR! 
npm ERR! ----------
npm ERR! 
npm ERR! Platform: darwin-x64 (22.4.0)
npm ERR! Cypress Version: 13.7.0
npm ERR! [FAILED] The Cypress App could not be downloaded.
npm ERR! [FAILED] 
npm ERR! [FAILED] Does your workplace require a proxy to be used to access the Internet? If so, you must configure the HTTP_PROXY environment variable before downloading Cypress. Read more: https://on.cypress.io/proxy-configuration
npm ERR! [FAILED] 
npm ERR! [FAILED] Otherwise, please check network connectivity and try again:
npm ERR! [FAILED] 
npm ERR! [FAILED] ----------
npm ERR! [FAILED] 
npm ERR! [FAILED] URL: https://download.cypress.io/desktop/13.7.0?platform=darwin&arch=x64
npm ERR! [FAILED] Error: Failed downloading the Cypress binary.
npm ERR! [FAILED] Response code: 502
npm ERR! [FAILED] Response message: Bad Gateway
npm ERR! [FAILED] 
npm ERR! [FAILED] ----------
npm ERR! [FAILED] 
npm ERR! [FAILED] Platform: darwin-x64 (22.4.0)
npm ERR! [FAILED] Cypress Version: 13.7.0

npm ERR! A complete log of this run can be found in: /Users/stefantegeltija/.npm/_logs/2024-03-19T14_38_18_486Z-debug-0.log

CloneOfAlex · 2024-03-19T14:45:50Z

OK - try other versions, example 13.3.3

StefanTegeltija · 2024-03-19T14:47:15Z

Man I tried all versions, and problem is still here, download my repo, try by yourself and I'm sure you will reproduce the issue immediately.

CloneOfAlex · 2024-03-19T14:48:20Z

run this and report back

npx cypress --version

CloneOfAlex · 2024-03-19T14:53:59Z

Man I tried all versions, and problem is still here, download my repo, try by yourself and I'm sure you will reproduce the issue immediately.

not willing to spend time debugging your repo
the issues you addressing non existent if someone uses latest node and Cypress version
using 1-3 years old components prone to issues (see change logs for node and cypress)

also even if i reproduce, what do you expect me to do? fix code utilizing old technology? that would be silly application of time

Your number 1 priority should be upgrading all of your dependencies to the latest, and next keeping your dependencies stack at latest stable versions at all times .. why? because it makes sense - Chrome, Edge, FireFox browsers keep upgrading and so is Cypress etc

StefanTegeltija · 2024-03-19T14:56:47Z

NO it would be proof that even with latest Node and latest Cypress version problem still exists.

CloneOfAlex · 2024-03-19T14:58:33Z

NO it would be proof that even with latest Node and latest Cypress version problem still exists.

sure hence invest your own time into upgrading your own repo and testing it there .. etc

StefanTegeltija · 2024-03-19T15:01:00Z

Problem is on Cypress side not on my side. So Cypress QAs has to test it, not myself (I did enough by creating reproducible example). If you aren't member of Cypress company ofc you shouldn't spend time on it, but they should.

CloneOfAlex · 2024-03-19T15:52:18Z

run this and report back

npx cypress --version

run this and please report back

npx cypress --version

StefanTegeltija · 2024-04-09T12:54:45Z

I finally found some time to update this repo: https://github.com/StefanTegeltija/memory-leak-in-long-running-single-test

Even I updated everything on the latest version, the issue still exist.

Please download repo and try it by yourself.

My Chrome version is: 123.0.6312.107

Steps:

run this: npm run open:dev
run 001_Casinos.feature

stefantegeltija@MTO-1416 memory-leak-in-long-running-single-test % npx cypress --version
Cypress package version: 13.7.2
Cypress binary version: 13.7.2
Electron version: 27.1.3
Bundled Node version: 18.17.1

John4516 · 2024-06-27T15:37:39Z

I still have the same issue with npx cypress run & open

I tested using differents method :

launchOptions.args.push('--max_old_space_size=xxxx');
launchOptions.args.push('--disable-dev-shm-usage');

Or adding :

numSnapshotsKeptInMemory = 1
experimentalMemoryManagement: true,

Or this method :

- launchOptions.args.push('--js-flags=--expose-gc');

afterEach(() => {
	cy.window().then(win => {
		// window.gc is enabled with --js-flags=--expose-gc chrome flag
		if (typeof win.gc === 'function') {
			// run gc multiple times in an attempt to force a major GC between tests
			win.gc();
			win.gc();
			win.gc();
			win.gc();
			win.gc();
		}
	});
});

I downgraded cypress version but nothing seems to be work.

It's rather frustrating that it crashes so quickly and so often.

Any updates for memory problem?

Cypress: 13.9.0
Browser: Chrome 118 (headless)
Node Version: v20.9.0

ludo550 · 2024-09-12T07:13:03Z

@John4516 and to anyone else still looking for a solution. My solution to this issue ultimately was the use of NO_COMMAND_LOG . In your cypress run command, you can add --env NO_COMMAND_LOG=1. Works like a charm for me. To use this with debugging, try using cypress-terminal-reporter or something similar or add custom loggers. I am using custom loggers as I didn't want to overpopulate my command line or terminal. You can also add, Cypress's garbage collection mechanisms as added benefit but I doubt you will need it. I anticipate your main leak comes from the command log itself which keeps building up in the browser, headed or headless. I doubt video recording will cause this but you can play around. Good luck!

jennifer-shehane added the type: performance 🏃‍♀️ Performance related label May 22, 2019

jennifer-shehane changed the title ~~Memory leak in long test runs~~ Memory leak in long running single test Jun 6, 2019

jennifer-shehane added the stage: ready for work The issue is reproducible and in scope label Jun 6, 2019

jennifer-shehane mentioned this issue Jun 6, 2019

'Aw, Snap' Error in Test Runner consistently occurs every minute when a test is running during cypress open #1937

Closed

brian-mann closed this as completed Jun 17, 2019

brian-mann added type: duplicate This issue or pull request already exists and removed stage: ready for work The issue is reproducible and in scope type: performance 🏃‍♀️ Performance related labels Jun 17, 2019

hadighattas mentioned this issue Sep 23, 2019

Error: Source and destination must not be the same. #4637

Closed

cypress-io locked as resolved and limited conversation to collaborators Jan 3, 2020

Konstruktour mentioned this issue Feb 5, 2020

Memory leak still in long running single test #6330

Closed

cypress-io unlocked this conversation Feb 5, 2020

jennifer-shehane removed the type: duplicate This issue or pull request already exists label Feb 13, 2020

jennifer-shehane reopened this Feb 13, 2020

jennifer-shehane added the type: performance 🏃‍♀️ Performance related label Feb 13, 2020

cypress-bot bot added the stage: needs investigating Someone from Cypress needs to look at this label Feb 13, 2020

nagash77 added the prevent-stale mark an issue so it is ignored by stale[bot] label Apr 3, 2023

jennifer-shehane mentioned this issue Oct 31, 2023

Huge Memory Usages #28184

Closed

jennifer-shehane added the Reproducible Can be reproduced label Jan 5, 2024

jennifer-shehane changed the title ~~Memory leak in long running single test~~ Memory leak in long running single test - crash Sep 16, 2024

Memory leak in long running single test - crash #4164

Memory leak in long running single test - crash #4164

Comments

Konstruktour commented May 8, 2019 • edited Loading

Current behavior:

Desired behavior:

Steps to reproduce: (app code and test code)

Versions

scharf commented Jun 16, 2019

brian-mann commented Jun 17, 2019

brian-mann commented Jun 17, 2019

brian-mann commented Jun 17, 2019

cypress-bot bot commented Jun 27, 2019

Konstruktour commented Nov 4, 2019 • edited Loading

Konstruktour commented Nov 8, 2019

jennifer-shehane commented Jan 3, 2020

jennifer-shehane commented Feb 5, 2020 • edited Loading

Konstruktour commented Feb 5, 2020

Konstruktour commented Feb 13, 2020

Oliboy50 commented Jul 18, 2020

jennifer-shehane commented Oct 8, 2020

pardamike commented Oct 9, 2020 • edited Loading

dmolin commented Mar 21, 2023 • edited Loading

krisodb commented Mar 21, 2023 • edited Loading

CloneOfAlex commented Jul 20, 2023

nagash77 commented Jul 25, 2023

StefanTegeltija commented Jan 4, 2024 • edited Loading

scharf commented Jan 4, 2024

CloneOfAlex commented Jan 4, 2024

StefanTegeltija commented Jan 5, 2024 • edited Loading

LukasLewandowski commented Mar 19, 2024

CloneOfAlex commented Mar 19, 2024 • edited Loading

StefanTegeltija commented Mar 19, 2024

CloneOfAlex commented Mar 19, 2024 • edited Loading

CloneOfAlex commented Mar 19, 2024

StefanTegeltija commented Mar 19, 2024

StefanTegeltija commented Mar 19, 2024 • edited by jennifer-shehane Loading

CloneOfAlex commented Mar 19, 2024

StefanTegeltija commented Mar 19, 2024

CloneOfAlex commented Mar 19, 2024

CloneOfAlex commented Mar 19, 2024

StefanTegeltija commented Mar 19, 2024

CloneOfAlex commented Mar 19, 2024

StefanTegeltija commented Mar 19, 2024

CloneOfAlex commented Mar 19, 2024

StefanTegeltija commented Apr 9, 2024 • edited Loading

John4516 commented Jun 27, 2024 • edited by jennifer-shehane Loading

ludo550 commented Sep 12, 2024 • edited Loading

Konstruktour commented May 8, 2019 •

edited

Loading

Konstruktour commented Nov 4, 2019 •

edited

Loading

jennifer-shehane commented Feb 5, 2020 •

edited

Loading

pardamike commented Oct 9, 2020 •

edited

Loading

dmolin commented Mar 21, 2023 •

edited

Loading

krisodb commented Mar 21, 2023 •

edited

Loading

StefanTegeltija commented Jan 4, 2024 •

edited

Loading

StefanTegeltija commented Jan 5, 2024 •

edited

Loading

CloneOfAlex commented Mar 19, 2024 •

edited

Loading

CloneOfAlex commented Mar 19, 2024 •

edited

Loading

StefanTegeltija commented Mar 19, 2024 •

edited by jennifer-shehane

Loading

StefanTegeltija commented Apr 9, 2024 •

edited

Loading

John4516 commented Jun 27, 2024 •

edited by jennifer-shehane

Loading

ludo550 commented Sep 12, 2024 •

edited

Loading