Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vitest runs tests 3x slower than Jest with threads: true (default) setting #579

Closed
6 tasks done
dmLinker opened this issue Jan 19, 2022 · 83 comments
Closed
6 tasks done

Comments

@dmLinker
Copy link

Describe the bug

With the latest Vitest/Vite dependencies, Vitest is ~3 times slower than Jest in https://github.com/EvHaus/jest-vs-jasmine (more or less like a typical front-end repo) with the threads: true (default) setting.

Benchmark 2: yarn workspace jest test
  Time (mean ± σ):     34.704 s ±  2.548 s    [User: 91.246 s, System: 16.712 s]
  Range (min … max):   32.017 s … 41.447 s    10 runs
 
Benchmark 3: yarn workspace vitest test --threads=true
  Time (mean ± σ):     99.077 s ±  2.015 s    [User: 185.493 s, System: 40.303 s]
  Range (min … max):   97.352 s … 103.406 s    10 runs

Related comment #229 (comment). That issue was closed but threads: true is still 3-4 slower than Jest after the issue was closed and the repro dependencies updated).

The issue was raised in Discord chat as well.

With threads: false Vitest is ~2 times faster, but the cost of it - there's no real isolation. With threads: false it's unfortunately not even close what Jest is doing in terms or resetting state, so not even sure if makes sense to compare threads: false with Jest (maybe threads: false should be compared with uvu or other "run-once-and-forget" runners with no real isolation). In a variety of projects that we have worked on, even with pure stateless components, the proper isolation is a super important concern. Especially in watch mode where introducing some unwanted state/mocking or polluting global/process state is simply an expected thing to happen due to the nature of various incremental changes (that are not always good or final) that simply break not properly isolated watch mode.

Reproduction

git clone https://github.com/EvHaus/jest-vs-jasmine
yarn

(if required, install hyperfine)

hyperfine --warmup 1 'yarn workspace jest test' 'yarn workspace vitest test --threads=true'

System Info

System:
    OS: macOS 11.2.3
    CPU: (16) x64 Intel(R) Xeon(R) W-2140B CPU @ 3.20GHz
    Memory: 25.41 MB / 32.00 GB
    Shell: 3.2.57 - /bin/bash
  Binaries:
    Node: 16.13.1 - ~/.nvm/versions/node/v16.13.1/bin/node
    Yarn: 3.1.1 - ~/.yarn/bin/yarn
    npm: 8.1.2 - ~/.nvm/versions/node/v16.13.1/bin/npm
    Watchman: 4.9.0 - /usr/local/bin/watchman
  Browsers:
    Chrome: 97.0.4692.71
    Firefox Developer Edition: 61.0
    Safari: 14.0.3

Used Package Manager

npm

Validations

@cbxp
Copy link

cbxp commented Jan 24, 2022

Vitest is also about 2-3x slower than @web/test-runner on the same project.

But also, it reports the time incorrectly: when it says that tests ran for 7 seconds, the time utility reports it was actually 25s of user time.

@Demivan
Copy link
Member

Demivan commented Jan 24, 2022

@cbxp User time will be higher because Vitest is multi-threaded.
User time is the time processor spent running a process. If you have multiple cores, user time will be higher for multi-threaded apps. Because it aggregates all the work that all cores did.

chaii3 pushed a commit to chaii3/vitest that referenced this issue May 13, 2022
A slash was missing in the close-tag.
@black7375
Copy link

black7375 commented Jun 14, 2022

One of the reasons why Jest is fast is cache.

I added yarn workspace jest test --no-cache to the example of jest-vs-jasmine.
(Vitest's --threads=true is default.)

>> hyperfine --warmup 1 'yarn workspace jasmine test' 'yarn workspace jest test' 'yarn workspace jest test --no-cache' 'yarn workspace jest test --shard' 'yarn workspace vitest test' 'yarn workspace vitest test --isolate=false'
Benchmark 1: yarn workspace jasmine test
  Time (mean ± σ):     10.317 s ±  1.826 s    [User: 12.119 s, System: 0.563 s]
  Range (min … max):    8.012 s … 13.697 s    10 runs
 
Benchmark 2: yarn workspace jest test
  Time (mean ± σ):     30.254 s ±  1.570 s    [User: 185.648 s, System: 9.170 s]
  Range (min … max):   28.122 s … 32.521 s    10 runs
 
Benchmark 3: yarn workspace jest test --no-cache
  Time (mean ± σ):     44.370 s ±  1.824 s    [User: 296.171 s, System: 10.601 s]
  Range (min … max):   42.537 s … 48.359 s    10 runs
 
Benchmark 4: yarn workspace jest test --shard
  Time (mean ± σ):     30.358 s ±  1.592 s    [User: 186.843 s, System: 9.127 s]
  Range (min … max):   28.141 s … 32.958 s    10 runs
 
Benchmark 5: yarn workspace vitest test
  Time (mean ± σ):     90.731 s ±  2.527 s    [User: 529.527 s, System: 36.283 s]
  Range (min … max):   87.438 s … 94.203 s    10 runs
 
Benchmark 6: yarn workspace vitest test --isolate=false
  Time (mean ± σ):     17.051 s ±  0.867 s    [User: 49.972 s, System: 2.401 s]
  Range (min … max):   15.555 s … 18.515 s    10 runs
 
Summary
  'yarn workspace jasmine test' ran
    1.65 ± 0.30 times faster than 'yarn workspace vitest test --isolate=false'
    2.93 ± 0.54 times faster than 'yarn workspace jest test'
    2.94 ± 0.54 times faster than 'yarn workspace jest test --shard'
    4.30 ± 0.78 times faster than 'yarn workspace jest test --no-cache'
    8.79 ± 1.58 times faster than 'yarn workspace vitest test'

Another benchmark.
mocha's parallel looks pretty fast, too.

https://medium.com/dailyjs/javascript-test-runners-benchmark-3a78d4117b4
image
image
image

@pmdarrow
Copy link

pmdarrow commented Jul 15, 2022

My team in the process of migrating a large test suite (6073 tests) from jest to vitest. With jest, the full suite runs in 3m47s. With the latest vitest (0.18.0) it runs in 10m7s. We really want to continue with vitest but this kind of a slow down is making it challenging. We don't want to disable isolation, as it's crucial for reliability in a test suite this large. Is there any plan to improve the performance vitest, or is jest doing something fundamentally different that vitest won't be able to replicate? @antfu @sheremet-va @Demivan

@Demivan
Copy link
Member

Demivan commented Jul 15, 2022

@pmdarrow I'm investigating switching to vm.runInNewContext from vm.runInThisContext. This, hopefully, will allow us to drop costly isolation using Workers. We will essentially have the same isolation as Jest.

But I did not have time to work on this recently (there is the war going on)

@sheremet-va
Copy link
Member

@pmdarrow I'm investigating switching to vm.runInNewContext from vm.runInThisContext. This, hopefully, will allow us to drop costly isolation using Workers. We will essentially have the same isolation as Jest.

I don't think we can do it without getting the same problems Jest has, because VM has terrible support for ESM. I already tried it.

@sheremet-va
Copy link
Member

My team in the process of migrating a large test suite (6073 tests) from jest to vitest. With jest, the full suite runs in 3m47s. With the latest vitest (0.18.0) it runs in 10m7s. We really want to continue with vitest but this kind of a slow down is making it challenging. We don't want to disable isolation, as it's crucial for reliability in a test suite this large. Is there any plan to improve the performance vitest, or is jest doing something fundamentally different that vitest won't be able to replicate? @antfu @sheremet-va @Demivan

You can get speed boost by disabling isolation. Bear in mind tho, that it will make globalThis object sharable between tests.

@Demivan
Copy link
Member

Demivan commented Jul 15, 2022

After investigating it more and checking with @sheremet-va i don't think it can be implemented with current state of vm module. Fixing all the issues will mean getting all Jest bugs too.

Lets hope ShadowRealm implementation will be better then vm module.

@pmdarrow
Copy link

pmdarrow commented Jul 21, 2022

So is the summary that we're stuck with the performance where it's at right now, unless we want to sacrifice isolation? If so, I think this should be published in the docs somewhere. I know the docs say disabling isolation can improve performance, but it doesn't communicate that comparable tools (i.e. Jest) are 3x faster even with isolation. It would help to know up front why vite is slower by default and what is blocking it from becoming faster (vm module bugs?)

@antfu
Copy link
Member

antfu commented Jul 22, 2022

Vitest runs tests 3x slower than Jest

That is an incorrect statement. It really deps on what and how your tests are written. You can't say a man walks faster than cars without the context (in a traffic jam or something it might be true). The reason why this is happening commonly due to you are using some giant dependencies, where in isolation, they will be executed multiple times for each tests. Vitest is on-demand like Vite, while Jest will bundle the code, so in some scenario it would be more efficient than Vitest.

While indeed there is some space for optimization like pre-bundle the deps, or waiting for vm to support ESM. But in general, it's still trade-offs between speed and correctness. For performance and also for good practice, I would suggest to make the tests as "unit" as possible, or mocking dependencies whenever you can.

@dmLinker
Copy link
Author

while Jest will bundle the code, so in some scenario it would be more efficient than Vitest.

Can you please elaborate a bit on how/where exactly Jest bundles anything? AFAIK Jest transformers are invoked lazily whenever a file is being loaded by other files during a test file execution.

it's still trade-offs between speed and correctness. For performance and also for good practice, I would suggest to make the tests as "unit" as possible, or mocking dependencies whenever you can.

The thing is that such "unit" tests (deeply mocked) tests are fast in both Jest and Vitest (the difference is really minor, and not always in favour of the same framework). However, in addition, Jest is isolating much better, so the trade-offs between speed and correctness case are relevant for Vitest only, while with Jest one can get both in this case.

Unfortunately the isolation with the use vm in Jest comes with the current ESM support limitations. Hopefully those are resolved in future so potentially both frameworks can benefit from it.

@thebuilder
Copy link

Is there anyway of debugging vitest performance? Maybe list what takes time on each run, or which dependencies are loaded?
After switching a React Testing Library application (built with Vite) from Jest to Vitest, the test time CI (Azure DevOps) went from around 50s with code coverage using Jest, to 2 minutes without code coverage in Vitest.
It could very well be related to dependencies we are using (like Material UI), but it's really hard to debug.

@Weetbix
Copy link

Weetbix commented Jul 27, 2022

Like @thebuilder and @pmdarrow we have also experienced a 3x slowdown of tests after migrating to vitest from jest, our large CI unit test suite now takes 15 mins instead of 5 when running on GHA.

@antfu
Copy link
Member

antfu commented Jul 27, 2022

Ok, I can have a look at this. Can anyone set up a minimal repo (2~3 test files, or the less possible) of how the slowdown is being observed (as I personally do not face such a slowdown when doing unit testing or testing Vue)? Thanks

@EvHaus
Copy link
Contributor

EvHaus commented Jul 27, 2022

@antfu Maybe https://github.com/EvHaus/jest-vs-jasmine can help. It's more than 2~3 test files, but I think you can easily just strip out the test suites and reduce them. The latest benchmark shows:

   73.626s ± 0.308s for 'yarn workspace jest test'
   232.393s ± 0.457s for 'yarn workspace vitest test'

@sheremet-va
Copy link
Member

sheremet-va commented Jul 27, 2022

Hm, looking at the repo, I wonder if the problem is in .module.less files 🤔 Currently Vitest doesn't process css files, but processes *.module.* by default (we have a PR to auto mock it in #1512). We can start from that 👀

Disabling css with css: false and returning proxy for .module.less:

// plugin.js
{
      name: "remove-css",
      enforce: "post",
      transform(code, id) {
        if (id.endsWith(".module.less")) {
          const code = `
          export default new Proxy(
            {},
            {
              get(_, style) {
                return style;
              },
            }
          );`;
          return {
            code: code,
          };
        }
      },
    },

I have this result on M1 Air 8 GB:

Benchmark 1: yarn workspace jasmine test
  Time (mean ± σ):      3.719 s ±  0.052 s    [User: 3.476 s, System: 0.238 s]
  Range (min … max):    3.651 s …  3.838 s    10 runs
 
Benchmark 2: yarn workspace jest test
  Time (mean ± σ):     10.829 s ±  0.395 s    [User: 56.573 s, System: 6.929 s]
  Range (min … max):   10.561 s … 11.898 s    10 runs
 
Benchmark 3: yarn workspace jest test --shard
  Time (mean ± σ):     11.316 s ±  0.445 s    [User: 59.148 s, System: 7.274 s]
  Range (min … max):   10.771 s … 12.197 s    10 runs
 
Benchmark 4: yarn workspace vitest test
  Time (mean ± σ):     34.201 s ±  0.425 s    [User: 176.166 s, System: 32.009 s]
  Range (min … max):   33.561 s … 34.940 s    10 runs
 
Benchmark 5: yarn workspace vitest test --isolate=false
  Time (mean ± σ):      6.413 s ±  0.885 s    [User: 16.657 s, System: 1.793 s]
  Range (min … max):    6.076 s …  8.927 s    10 runs
 
  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet PC without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
 
Summary
  'yarn workspace jasmine test' ran
    1.72 ± 0.24 times faster than 'yarn workspace vitest test --isolate=false'
    2.91 ± 0.11 times faster than 'yarn workspace jest test'
    3.04 ± 0.13 times faster than 'yarn workspace jest test --shard'
    9.20 ± 0.17 times faster than 'yarn workspace vitest test'

Maybe you should also benchmark --no-threads, btw?

Considering CSS, I don't see a lot of changes unfortunately. So It seems the problem lies with how tinypool fires up isolated workers.

@pmdarrow
Copy link

pmdarrow commented Aug 4, 2022

@antfu is https://github.com/EvHaus/jest-vs-jasmine sufficient for a reproduction or would you like it to be trimmed down further?

@antfu
Copy link
Member

antfu commented Aug 4, 2022

I trimmed a bit locally. Flow + Less is a bit less common tbh, but yes with your tests I can confirm it surfaced some bottleneck of Vitest, where I am still investigating.

@pmdarrow
Copy link

@antfu is there anything that we could do to help move this forward?

@phifa
Copy link

phifa commented Sep 14, 2022

I am not sure my issue is related, but from time to time this happens, my fans are going off and cpu usage rises to 700%... Something going on with vitest... If this is related, how can I debug, mitigate this?

vitest_cpu_perf

@pmdarrow
Copy link

@phifa I don't think that's related to this discussion - that's just vitest using as many cores as possible. If you don't want this behaviour, you can turn it off: https://vitest.dev/guide/features.html#threads.

@MarkLyck
Copy link

Same experience here. Jest ran my test-suite in about 3-5 minutes, after migrating to vitest it now takes 10-17 minutes 😞

I'm sure it would be a lot faster if I disabled threads, but it's not worth dealing with all the potential issues that comes with that.

I do have .less files in my project as well, but they are not called .module.less so I'm not sure if that would affect anything?

@sheremet-va
Copy link
Member

Same experience here

I wonder, if enabling deps.fallbackCJS helps?

@ild0tt0re
Copy link

ild0tt0re commented Oct 1, 2022

@dmLinker @pmdarrow We also want to use vitest as our modern test tool but for now we are using @swc/jest with good performance gain (3x)

@moosemanf
Copy link

moosemanf commented Mar 17, 2023

for me it seems to be the mentioned barrel exports, that then hit all imports... so am I mistaken that the new deps.experimentalOptimizer option could help? I'm a bit lost on the synthax here, tryed via vite docs to add a

      deps: {
        experimentalOptimizer: {
          include: ['@@/tests/utils'],
          enabled: true,
        },

but that brought only a big go error that had an overflow in my console...
any ideas?

PS: Big, big thanks to the debug options, that is an enabler!!

@asdfjkalsdfla
Copy link
Contributor

asdfjkalsdfla commented Mar 18, 2023

@AriPerkkio Really great to see the performance improvements from #3006. Moving the 5000 test above from 5667.68s to 4.63s is huge! Thanks for the effort on that.

@sheremet-va
Copy link
Member

I've noticed that one of the reasons why there is a difference between Jest and Vitest is the fact that Vitest imports jsdom for each test file which takes a lot of time (300ms on my M1 Mac to simply import jsdom). So one of my recommended solutions is to try to use happy-dom, which takes 3 times less to import.

@adrian-gierakowski
Copy link

Vitest imports jsdom for each test file

I guess it wouldn’t do it when testing pure node code? Does it mean that the issues discussed here do not apply to testing server code?

@purepear
Copy link

purepear commented Apr 8, 2023

@adrian-gierakowski

Yep, it depends on the environment. You can check out my previous comment #579 (comment)

@thebuilder
Copy link

I've noticed that one of the reasons why there is a difference between Jest and Vitest is the fact that Vitest imports jsdom for each test file which takes a lot of time (300ms on my M1 Mac to simply import jsdom). So one of my recommended solutions is to try to use happy-dom, which takes 3 times less to import.

Uff, that's a lot. I guess when you combine this with testing library, it really adds up.

@redbar0n
Copy link

Why not import jsdom or happy-dom once and reuse for all test files in need of it?

@sheremet-va
Copy link
Member

Why not import jsdom or happy-dom once and reuse for all test files in need of it?

Each test file runs inside a separate worker. You cannot transfer JSDOM constructor between worker threads as far as I know.

@pkolt
Copy link

pkolt commented Apr 16, 2023

Hi guys!

Our project has a lot of files that are connected during the test (utils, components, openApi interfaces). Because of what, even one test takes a very long time (threads: true, isolate: true):

Duration  12.96s (transform 8.48s, setup 859ms, collect 10.01s, tests 528ms, environment 1.11s, prepare 151ms)

I run the test in debug mode with DEBUG=vite-node:* vitest and I see that vitest connects all the files one by one, it takes a lot of time:

vite-node:server:request /src/store/rootReducer.ts +43ms
2023-04-16T12:55:51.351Z vite-node:client:execute /home/pkolt/projects/moto_ui/src/store/rootReducer.ts
  vite-node:server:request /src/store/actionSession/index.ts +23ms
2023-04-16T12:55:51.357Z vite-node:client:execute /home/pkolt/projects/moto_ui/src/store/actionSession/index.ts
  vite-node:server:request /src/store/actionSession/slice.ts +6ms
2023-04-16T12:55:51.363Z vite-node:client:execute /home/pkolt/projects/moto_ui/src/store/actionSession/slice.ts
  vite-node:server:request /src/store/actionSession/types.ts +7ms
2023-04-16T12:55:51.369Z vite-node:client:execute /home/pkolt/projects/moto_ui/src/store/actionSession/types.ts
  vite-node:server:request /src/store/actionSession/thunks.ts +4ms
2023-04-16T12:55:51.378Z vite-node:client:execute /home/pkolt/projects/moto_ui/src/store/actionSession/thunks.ts
  vite-node:server:request /src/store/bankAccount/index.ts +10ms
2023-04-16T12:55:51.383Z vite-node:client:execute /home/pkolt/projects/moto_ui/src/store/bankAccount/index.ts
  vite-node:server:request /src/store/bankAccount/slice.ts +5ms
2023-04-16T12:55:51.389Z vite-node:client:execute /home/pkolt/projects/moto_ui/src/store/bankAccount/slice.ts
  vite-node:server:request /src/store/bankAccount/types.ts +7ms
2023-04-16T12:55:51.394Z vite-node:client:execute /home/pkolt/projects/moto_ui/src/store/bankAccount/types.ts
  vite-node:server:request /src/store/bankAccount/thunks.ts +3ms

Is it possible to merge the imported files into one so that Vitest does not waste time downloading them one by one?

@TheJaredWilcurt
Copy link
Contributor

SUMMARY

I've written up a summary of all the advice given in this GitHub Issue as a time saver for those coming here in the future.

Improving Vitest Performance

@sheremet-va
Copy link
Member

SUMMARY

I've written up a summary of all the advice given in this GitHub Issue as a time saver for those coming here in the future.

Improving Vitest Performance

--no-threads is outdated. If you want your tests to run in sequence without firing up new workers use --single-thread

--no-threads now uses child_process instead of workers, so your code might run even slower with it.

@luizeboli
Copy link

luizeboli commented Jun 2, 2023

Using v0.31.4.

The situation with me seems a bit worse than what I read here. I have 55 tests spread across 19 files. On my machine, these tests run in 28 seconds. However, on Azure DevOps (Pipeline), they take over 30 minutes (I gave up waiting).

I tried using the deps.experimentalOptimizer option, but all the tests fail with the error @vitejs/plugin-react can't detect preamble. Something is wrong.

With the threads: false or isolation: false option, several tests start to randomly break. (may be related to #1430 due RTL)

I'm using tools like testing-library, msw, material-ui, react-hook-form.

@black7375
Copy link

If a test that takes 30 seconds locally takes over 30 minutes on your server, the first thing you need to check is that the test is written correctly.
There seems to be something causing an infinite loop or timeout.

@luizeboli
Copy link

If a test that takes 30 seconds locally takes over 30 minutes on your server, the first thing you need to check is that the test is written correctly. There seems to be something causing an infinite loop or timeout.

After your comment, I realized there was a test with using vi.useFakeTimers, but it's strange that I didn't have any issues locally.

I ended up being able to make it work without any changes to the tests, using the option threads: false thanks to the issue I mentioned earlier.

@hugw
Copy link

hugw commented Jul 17, 2023

I also noticed that after moving from Jest to Vitest, my tests were taking 2-3x more time to run.

After doing some experiments lately, one thing that drastically reduced the time to run all specs was to disable experimental fetch (NODE_OPTIONS='--no-experimental-fetch' yarn test). We currently use whatwg-fetch.

Before: 126.28s
After disabling experimental fetch: 53.05s

@sheremet-va
Copy link
Member

Vitest 0.34.0 is released with the new experimentalVmThreads flag which should make tests run as fast as they do in Jest. If you have any issues with the new pool, please open a separate issue instead of commenting here.

@EvHaus
Copy link
Contributor

EvHaus commented Aug 2, 2023

I updated the benchmark to use experimentalVmThreads, the results confirm the optimizations!

vitest@0.34.1

'yarn workspace bun test' ran
    1.37 ± 0.06 times faster than 'yarn workspace vitest test --isolate=false'
    1.72 ± 0.07 times faster than 'yarn workspace jasmine test'
    6.49 ± 0.[34](https://github.com/EvHaus/test-runner-benchmarks/actions/runs/5730584383/job/15529649956#step:6:35) times faster than 'yarn workspace jest test'
   11.50 ± 0.65 times faster than 'yarn workspace vitest test'

vitest@0.34.1 with experimentalVmThreads:true

'yarn workspace bun test' ran
  1.51 ± 0.05 times faster than 'yarn workspace jasmine test'
  5.53 ± 0.05 times faster than 'yarn workspace jest test'
  5.60 ± 0.05 times faster than 'yarn workspace vitest test --isolate=false'
  5.61 ± 0.05 times faster than 'yarn workspace vitest test'

The test previous ran in 101.259s but now runs in 60.171s, which is also identical to Jest's speed!

Side-note: Looks like using isolate:true together with experimentalVmThreads:true isn't a good idea (it makes the tests run much slower). I'm guessing it re-enables isolation due to threads being used.

I think based on this excellent new development we can probably close this ticket?

@sheremet-va
Copy link
Member

Looks like using isolate:true together with experimentalVmThreads:true isn't a good idea

isolate should have zero effect on vmThreads because they are always isolated (one context per test file), so I'm not sure what you mean 🤔

The fastest way to run tests is still using only --threads --isolate=false because there is only a single cache for all test files.

I think based on this excellent new development we can probably close this ticket?

We still need a way to run tests using VM inside the forked process to support process.chdir and test suites that fail when running inside worker because they rely on unstable low-level API. This is the next step.

@EvHaus
Copy link
Contributor

EvHaus commented Aug 2, 2023

isolate should have zero effect on vmThreads because they are always isolated (one context per test file), so I'm not sure what you mean

My bad. I meant isolate:false. I noticed that if you do isolate:false together with experimentalVmThreads:true you don't get the speed benefits of disabling isolation. So I'm guessing experimentalVmThreads:trueignores the isolate config value entirely.

@sheremet-va
Copy link
Member

So I'm guessing experimentalVmThreads:trueignores the isolate config value entirely.

Yes, it has no effect: https://vitest.dev/config/#isolate

@sheremet-va
Copy link
Member

sheremet-va commented Aug 2, 2023

Conceptually, experimentalVmThreads is a separate runner (they are called pools in the codebase historically) from threads. We have several of them (we should probably simplify them into a single property):

  • threads
  • child_process
  • browser
  • experimentalVmThreads

isolate only has an effect on the first three. The last one is always isolated because otherwise it becomes just a threads. If you need to disable isolation, do not use experimentalVmThreads.

@karasavm
Copy link

karasavm commented Aug 2, 2023

We are using msw on React test and we are facing this issue when enabling experimentalVmThreads.


ReferenceError: Request is not defined
    at FetchInterceptor.<anonymous> (/Users/mike/devel/web-app/node_modules/@mswjs/interceptors/src/interceptors/fetch/index.ts:42:13)
    at step (/Users/mike/devel/web-app/node_modules/@mswjs/interceptors/lib/interceptors/fetch/index.js:59:23)
    at Object.next (/Users/mike/devel/web-app/node_modules/@mswjs/interceptors/lib/interceptors/fetch/index.js:40:53)
    at /Users/mike/devel/web-app/node_modules/@mswjs/interceptors/lib/interceptors/fetch/index.js:34:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/mike/devel/web-app/node_modules/@mswjs/interceptors/lib/interceptors/fetch/index.js:30:12)
    at globalThis.fetch (/Users/mike/devel/web-app/node_modules/@mswjs/interceptors/src/interceptors/fetch/index.ts:41:42)
    at baseHttpService (/Users/mike/devel/web-app/src/utils/services/api.service.ts:70:35)
    at Object.post (/Users/mike/devel/web-app/src/utils/services/api.service.ts:91:5)
    at Module.queryFunctions (/Users/mike/devel/web-app/src/utils/services/api-query-functions.ts:155:56)

@sheremet-va
Copy link
Member

We are using msw on React test and we are facing this issue when enabling experimentalVmThreads.

I will ask this again: If you have any issues with the new pool, please open a separate issue instead of commenting here. In your case, jsdom doesn't provide Request, so it's not available. When not using --experimentalVmThreads, we don't isolate Node.js globals from JSDOM globals, they are just overridden on top of already existing Node.js globals. You need to provide these globals yourself when using --experimentalVmThreads.

I am locking this issue. Open new issues if you have problems when running --experimentalVmThreads with a reproduction.

@vitest-dev vitest-dev locked as resolved and limited conversation to collaborators Aug 2, 2023
@sheremet-va sheremet-va removed the help wanted Extra attention is needed label Feb 12, 2024
@sheremet-va
Copy link
Member

sheremet-va commented Feb 15, 2024

vmThreads is out of experimental since Vitest 1.0, so I feel like this issue can be closed. A few suggestions on how you can improve the performance of your tests:

  • disable isolation if you don't need test-file isolation (Vitest won't reset module cache and changed global variables) - combine this with --poolOptions.threads.singleThread or --poolOptions.forks.singleFork to run all tests inside the same worker one after another which removes startup time completely
  • disable parallelism if your machine is not powerful enough
  • import functions directly from the module instead of a barrel file if you have one (don't use barrel files in general)
  • cleanup your setup files - remove imports that are non needed for every test
  • if you use a UI library with a lot of components, enable deps.optimizer.web.enabled and add it to deps.optimizer.web.include
  • try --pool=vmThreads or --pool=vmForks if your tests were faster in Jest

More tips from @TheJaredWilcurt: https://dev.to/thejaredwilcurt/improving-vitest-performance-42c6

@sheremet-va sheremet-va closed this as not planned Won't fix, can't repro, duplicate, stale Feb 15, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
Archived in project
Development

No branches or pull requests