[Bug]: Jest increase heap usage after each test

[max10rogerio]

Version

29.5.0

Steps to reproduce

• Create a suite with many tests using react (react-testing-library)
• Add args to jest command: --detectOpenHandles --logHeapUsage --forceExit --clearMocks
• Show heap usage increase after each test

Expected behavior

Jest not use a lot of memory

Actual behavior

When I run tests with --logHeapUsage then starts using ~130MB of memory and after some tests the heap usage increase between 2GB and 3GB...

command:

craco test --watchAll=false --detectOpenHandles --logHeapUsage --forceExit --clearMocks

This test uses ~1500MB of memory

But when I run him specifically the heap use is ~150MB

How to find the problem and solve it?

Additional context

Related:

#11956

I tried to use workerIdleMemoryLimit but not works

Docs:

https://jestjs.io/docs/next/configuration#workeridlememorylimit-numberstring

Command:

craco test --watchAll=false --coverage --detectOpenHandles --logHeapUsage --forceExit --clearMocks --workerIdleMemoryLimit=500MB

Result:

Environment

System:
    OS: Linux 6.2 Pop!_OS 22.04 LTS
    CPU: (12) x64 AMD Ryzen 5 5600X 6-Core Processor
  Binaries:
    Node: 18.15.0 - ~/.nvm/versions/node/v18.15.0/bin/node
    Yarn: 1.22.19 - ~/.nvm/versions/node/v18.15.0/bin/yarn
    npm: 9.5.0 - ~/.nvm/versions/node/v18.15.0/bin/npm
  npmPackages:
    jest: ^29.5.0 => 29.5.0

[max10rogerio]

I read this commend and I used this node option: --no-compilation-cache

#11956 (comment)

I tried this command:

node --expose-gc --no-compilation-cache ./node_modules/.bin/craco test --watchAll=false --coverage --maxWorkers=4 --logHeapUsage

This command reduce memory consumption of ~2GB ~3GB to ~1GB (max)

But the memory leak problem persist:

See this test:

Jest says use 1GB when running with all tests

But running alone, Jest says use 95MB

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[max10rogerio]

....Idk

[GBPROB2]

See #11956

By the looks of it, the issue is related to node rather then Jest itself. So don't worry, you're not the only one with this issue

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[SMH110]

We are having the same issue, our ~2000 tests start to take ~18 minutes and hit OOM
I took few snapshots and noticed this

It seems to me that everything been loaded many times, not sure why?

Is this related ?

[ricardodnb]

I have the same issue as @SMH110 running a suite alongisde supertest. Running each test individually uses ~140MB but when running the entire suite it goes to like 6GB until crashing. By looking at heap inspection i see up to 4GB of compiled (with babel-jest) code as string. Why jest compiles the same files over and over again and they are not cleared?

[JosefBredereck]

This issue is bothering us too. We are constantly overflowing our build server because the test files do not get cleared.

[daraclare]

This is also causing us issues, we still have the issue after updating to node 18, which we had hoped would fix the issue.

[maximelafarie]

I'm also having trouble with jest increasing heap memory.
I'm trying to debunk this with the Chrome Node inspector as explained here, if it helps someone...

You also can use the flag --detectLeaks (beta feature) in the jest command, it will make the test crash if it detects any memory leak, but tbh the logs are quite poor.

[ctsstc]

When jest starts spinning off multiple nodes and they're all eating up the RAM it makes my machine come to a crawl. I've also been experiencing random hangs. Our test suite of nearly 2k tests used to be able to run in 35-55 seconds averaging about 45 seconds, my current test hung and then crawled and took 350 seconds :\

I imagine part of this is due to the RAM, I'm on an M1 Macbook Pro with 16GB, but when these tests start taking up tons of RAM it starts to hit the page swap and compression and slows everything down. It's not the RAMs fault per-say but a side effect of the tests taking up way too much RAM. I shouldn't see my test suite spiking up 8-16GB of RAM when I run it.

When I measure the memory heap it will get up to about 2gb then seems to clean up and drop down to 1.5gb and climbs back up again. This is when it's in a single thread though. Once it's running there are several node instances using up that much each.

We're on:
node: v18.16.0
jest: 29.6.1

@maximelafarie I was just reading that article yesterday, but haven't had time to dig into this, just checked out the heap flag and noticed it growing, dropping (likely GC), and growing again.

This hanging is nuts:

Edit: tried locking in: "jest": "~29.4.3" ran: npx jest --runInBand --logHeapUsage still saw it spiking up to about 2GB on the heap again.

[ctsstc]

I've switched over to @swc/jest from ts-jest and it seems like all my problems have gone away, so I think a lot of my issues may be a transpilation issue coming from ts-jest. Overall I do still think my RAM consumption still seems to be overkill, but it does seem to be doing better. I'll have to dig into that article and see if I can track down what's causing the consumption.

Been stuck trying to get our coverage to work with @swc/jest so I might be stuck on ts-jest for now.
I did find this relevant issue it seems: kulshekhar/ts-jest#1967

This comment kulshekhar/ts-jest#1967
Brings up a jest custom runtime which seemed to help a bit with memory consumption, but not my overall slowness and hesitation for my tests to even start up.
https://github.com/reside-eng/jest-runtime

Edit/Update: We switched over to @swc/jest and our test suite is solid and running in about 16 seconds now 🎉

Also found that the coverage shenanigans were patched in ts-jest here:

• fix for decorator coverage kulshekhar/ts-jest#488

And found the following open bugs with a proposed similar fix:

• Incorrect Jest coverage  swc-project/swc#3854

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[beingfranklin]

I read this commend and I used this node option: --no-compilation-cache

✅ This worked for us. It was passing on our local machine but was failing in our pipelines on Azure, even after allocating more resources.

We suspect that one of the causes of our issue was the use of Inversify. During each test, we had to spin up and rebind containers. Using the --no-compilation-cache node option resolved our issue for the time being.

[kiranmurali93]

We had the same issue so I did a work around until this issue gets fixed,
https://blog.kiranpk.dev/navigating-the-jest-memory-issue-my-workaround

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[SimenB]

I assume this is a duplicate of #11956.

If not please open up a new issue with a minimal reproduction.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[github-actions]

This issue was closed because it has been stalled for 30 days with no activity. Please open a new issue if the issue is still relevant, linking to this one.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.