TestContext scope is not as documented, broken examples

[UnaiUribarri-TomTom]

Describe the bug

https://vitest.dev/guide/test-context.html states that TestContext is setup before each test function and cleaned up after each test function, as documented in the sample code copy & pasted from there.

// my-test.ts
import { test } from 'vitest'

const todos = []
const archive = []

export const myTest = test.extend({
  todos: async ({ task }, use) => {
    // setup the fixture before each test function
    todos.push(1, 2, 3)

    // use the fixture value
    await use(todos)

    // cleanup the fixture after each test function
    todos.length = 0
  },
  archive
})

// xx.test.ts
import { expect } from 'vitest'
import { myTest } from './my-test.ts'

myTest('add items to todos', ({ todos }) => {
  expect(todos.length).toBe(3)

  todos.add(4) /// ADDED BY ME. This needs to be replaced by `todos.push(4)`
  expect(todos.length).toBe(4)
})

myTest('move items from todos to archive', ({ todos, archive }) => {
  expect(todos.length).toBe(3) /// ADDED BY ME. This fails because todos.length is 4. 
  expect(archive.length).toBe(0)

  archive.push(todos.pop())
  expect(todos.length).toBe(2)
  expect(archive.length).toBe(1)
})

After fixing the obvious error in the first test (todos.add is not defined, replaced by todos.push), the second test fails because the context is not reset between test executions as documented. Instead, a single TextContext is shared between both test executions and the cleanup is only executed after both tests are done.

Reproduction

• Copy & paste the example files
• Run vitest run xx
• Output:

 FAIL  src/t/xx.test.ts > move items from todos to archive
AssertionError: expected 4 to be 3 // Object.is equality

- Expected
+ Received

- 3
+ 4

 ❯ src/t/xx.test.ts:12:24
     10|
     11| myTest('move items from todos to archive', ({ todos, archive }) => {
     12|   expect(todos.length).toBe(3)
       |                        ^
     13|   expect(archive.length).toBe(0)
     14|

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯

 Test Files  1 failed (1)
      Tests  1 failed | 1 passed (2)
   Start at  22:48:08
   Duration  190ms (transform 33ms, setup 0ms, collect 25ms, tests 6ms, environment 0ms, prepare 47ms)

System Info

System:
    OS: macOS 13.6
    CPU: (10) arm64 Apple M1 Max
    Memory: 137.77 MB / 64.00 GB
    Shell: 3.5.1 - /opt/homebrew/bin/fish
  Binaries:
    Node: 20.8.1 - /opt/homebrew/bin/node
    Yarn: 1.22.19 - /opt/homebrew/bin/yarn
    npm: 10.1.0 - /opt/homebrew/bin/npm
    Watchman: 2023.10.09.00 - /opt/homebrew/bin/watchman
  Browsers:
    Chrome: 118.0.5993.117
    Safari: 16.6

Used Package Manager

npm

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[sheremet-va]

What version of Vitest are you using?

[denisw]

For the record, as I was confused why #4410 "fixes" the issue experienced by @UnaiUribarri-TomTom: the failure to clean up was probably an instance of issue #4403, which is already fixed in the main branch but not in the latest release at the time of writing (v0.34.6).