🚀 Feature: when tests fail, say "I expect these new values" and replace my expected values with the actual ones (snapshots)

[a-roberts]

Feature Request Checklist

• I have read and agree to Mocha's Code of Conduct and Contributing Guidelines
• I have searched for related issues and issues with the faq label, but none matched my issue.
• I want to provide a PR to resolve this

Overview

Hello, I'm using Mocha for plenty of tests involving lots of data from Prometheus. I'm querying the data, formatting it, and providing API responses.

Sometimes I'll make an adjustment to the code and the tests will fail for good reason - but then I have to copy the new results out of the diff and say yep, these values are fine, accept them. A good reason might be: I now want 10 arrays of data instead of 5 so I can provide more accuracy, but my tests expect five (with exact values in them).

I'd like a flag like --accept-incoming (or similar, so long as it exists) so that when I get a failure like (my big array is not deeply equal to my big other array) - I can just say: yep, I know, and I want my expected response to now be that.

Instead I'm here deleting the expected response object's data, copying that out of the diff, JSON formatting it in VScode, and pasting it in. Fine, but I've got hundreds of tests that rightfully fail as I'm getting different (but better) values out.

Can Mocha do this for me already? Have I missed a trick? Cheers

Suggested Solution

Add an --accept-incoming flag so that when Mocha test(s) fail, instead of me manually fixing all of my assertions, the values will "simply" be replaced with what it now sees.

In your package.json scripts you might have:
npm t -> run your mocha tests, great
npm t:fix could be -> go ahead and accept all of those failures by making my assertions match what is now found

Alternatives

Doing it manually - it's a pain with vast amounts of data

[JoshuaKGoldberg]

This is a good idea! So good, in fact, other test runners have adopted it under the name of snapshots, or snapshot testing. Two examples:

• Snapshot Testing · Jest
• Snapshot | Guide | Vitest

It's a bit out of scope for Mocha for now. We see Mocha as being just a foundational, straightforward test framework that doesn't encompass the transpiling and file system writing that snapshot testing entails. You're better off using a more rich framework such as Jest or Vitest.

#5027 has more details on how we see Mocha's position now. Thanks for asking! 🤎