cy.task('combineCoverage') timed out after waiting 60000ms getting this error during bitbucket CI build.

[Ppkd2021]

Logs and screenshots
cy.task('combineCoverage') timed out after waiting 60000ms. getting this error a lot during the execution of test cases on bitbucket but locally when the same test cases are executed then it is working fine.
The above error is randomly thrown for various test cases.

Versions
Version - 3.9.2
Cypress version - 7.3.0

• What is this plugin's version? If this is NOT the latest released version can you try the latest version, please?
• If the plugin worked before in version X, but stopped after upgrading to version Y, please try the released versions between X and Y to see where the breaking change was.
• What is Cypress version?
• What is your operating system?
• What is the shell?
• What is the Node version?
• What is the NPM version?
• How do you instrument your application? Cypress does not instrument web application code, so you need to do it yourself.
• When running tests, if you open the web application in regular browser, and open DevTools, do you see window.__coverage__ object? Can you paste a screenshot?
• Is there .nyc_output folder? Is there .nyc_output/out.json file. Is it empty? Can you paste at least part of it so we can see the keys and file paths?
• Do you have any custom NYC settings in package.json (nyc object) or in other NYC config files
• Do you run Cypress tests in a Docker container?

Describe the bug
A clear and concise description of what the bug is.

Link to the repo
Bugs with a reproducible example, like an open source repo showing the bug, are the most likely to be resolved.

Example

[Ppkd2021]

@bahmutov Please can you help us in resolving this issue as due to this it is costing us a lot of build time in bitbucket, If one test case fails due to the above error then we need to re-run all the test cases in bitbucket that is another challenge

[bahmutov]

Can you create a reproducible example for me to debug this?

…

Sent from my iPhone

On Jun 16, 2021, at 07:59, Ppkd2021 ***@***.***> wrote:  @bahmutov Please can you help in resolving this issue as due to this it is costing us a lot of build time in bitbucket. If one test case fails due to the above error then we need to re-run all the test cases in bitbucket that is another challenge. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[Ppkd2021]

@bahmutov The issue is not happening with one or two specific test cases, It is random in nature.
.

[dvkruchinin]

Hi @bahmutov,
I sometimes encounter such an error in GitHub Actions CI. Can you please suggest the debug settings when running Cypress tests to get the correct output in case of an error?

[bahmutov]

Enable the debut logs via an environment variable DEBUG?

…

Sent from my iPhone

On Jul 20, 2021, at 10:45, Dmitry Kruchinin ***@***.***> wrote:  Hi @bahmutov, I sometimes encounter such an error in GitHub Actions CI. Can you please suggest the debug settings when running Cypress tests to get the correct output in case of an error? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[dvkruchinin]

Enable the debut logs via an environment variable DEBUG?
…
Sent from my iPhone
On Jul 20, 2021, at 10:45, Dmitry Kruchinin @.***> wrote:  Hi @bahmutov, I sometimes encounter such an error in GitHub Actions CI. Can you please suggest the debug settings when running Cypress tests to get the correct output in case of an error? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

Yes. Probably the inclusion of a full debug will not give the necessary result?
Or need to use DEBUG=cypress:*?

[bahmutov]

Look at the debug section in the readme

…

Sent from my iPhone

On Jul 20, 2021, at 10:58, Dmitry Kruchinin ***@***.***> wrote:  Enable the debut logs via an environment variable DEBUG? … Sent from my iPhone On Jul 20, 2021, at 10:45, Dmitry Kruchinin @.***> wrote:  Hi @bahmutov, I sometimes encounter such an error in GitHub Actions CI. Can you please suggest the debug settings when running Cypress tests to get the correct output in case of an error? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Yes. Probably the inclusion of a full debug will not give the necessary result? Or need to use DEBUG=cypress:*? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[dvkruchinin]

Look at the debug section in the readme
…
Sent from my iPhone
On Jul 20, 2021, at 10:58, Dmitry Kruchinin @.> wrote:  Enable the debut logs via an environment variable DEBUG? … Sent from my iPhone On Jul 20, 2021, at 10:45, Dmitry Kruchinin @.> wrote:  Hi @bahmutov, I sometimes encounter such an error in GitHub Actions CI. Can you please suggest the debug settings when running Cypress tests to get the correct output in case of an error? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Yes. Probably the inclusion of a full debug will not give the necessary result? Or need to use DEBUG=cypress:*? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

The error was reproduced. But the reason is still not clear.
https://github.com/dvkruchinin/cvat/pull/156/checks?check_run_id=3120924930

[chasemgray]

@bahmutov We stopped using cypress code coverage due to this issue. What we found is that on longer tests, the method which stringifies and parses the code coverage stats can get slower and slower and eventually time out.

[bahmutov]

:(

…

Sent from my iPhone

On Jan 6, 2022, at 19:56, Chase Gray ***@***.***> wrote:  @bahmutov We stopped using cypress code coverage due to this issue. What we found is that on longer tests, the method which stores and retrieves the code coverage stats can get slower and slower and eventually time out. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[Fuhji]

We are having the same issue, any workarounds re: strinfigy are out of date. Unless theres some workaround or fix we'll also have to stop using cypress code coverage

[raysunqi]

+1
having the same issue on gitlab ci builds with 4gb ram, local works fine, increasing 'taskTimeout' to 5 mins doesn't help

[dane-harnett]

We have also been experiencing this issue and my investigation found the following:

Commenting out (and adjusting the variable names) the following lines seemed to resolve the issue:
https://github.com/cypress-io/code-coverage/blob/master/support.js#L17-L18

These lines filter out the coverage results from the spec/test files.

For some reason these lines get slower and slower each time they process the coverage for each test in our suite.

For reference our resulting coverage output when stringified (excluding the spec/test files) is about 1MB.

Some potential solutions that I can think of:

1. Add a config option to skip stripping out the spec/test file results
2. Improve the performance of the filter functions

[dane-harnett]

For some more info:

Our coverage object contains ~4500 keys.
Each check by minimatch takes about 3.5ms. So in total it almost takes 20s for this part of the process.

I'm unsure why it eventually gets to >60s but after about three tests it will eventually fail.

[eulersson]

I'm facing the same issue as well. How can I provide more information like the number of keys, when running from CI?

I think I have 244 keys and this problem happens on my AWS EC2 T3 Medium (4 GB RAM).

If I cy.pause() when running locally and then check the coverage object I get 244, I don't know if this is relevant, compared to @dane-harnett 's 4500 it's nothing.

Object.keys(window.__coverage__).length
244

This error only happens on CI, never seen it when running it locally.

[eulersson]

Here's a bit of an update that seemed to work for me, but sadly on the resources side, not the plugin.

I launched 3 pipelines in GitLab and they all passed with coverage enabled after changing the amount of CPU my docker runner uses.

My runner (T3 Medium on AWS which as 2 vCPU and 4GB RAM) was limited to use 1 CPU in the GitLab Runner configuration because I wanted to have room for SSHing and debugging, but now I increased it to 2, that is, the full compute power of the machine, and all pipelines passed.

So be aware that the issue might be related to CPU and not so much about RAM.

Those are my speculations though...

[eulersson]

False positive. I'm still getting those errors again. I'm using Nx for my project management.

[artipsingh]

Is there any update on this issue? we've started to experience this error during deployments on our CI runs

[aantes-st]

For me this task fails when the coverage object is too large. The task never makes it across to the node process, leading me to believe there is a limit to the size of the string you can pass through cy.task. I wonder if this could be updated to send coverage in a series of batches?