Unexpected Behaviour with vm.expectRevert()

[Equious]

To summarize - vm.expectRevert(bytes("Custom Error Message")) will pass a test regardless of if the custom error message actually matches.

Contract function for reference:

function fund() public payable {
        require(msg.value >= MINIMUM_USD, "You need to spend more ETH!");
        s_addressToAmountFunded[msg.sender] += msg.value;
        s_funders.push(msg.sender);
    }

Test being run:

function testFailsWithoutEnoughETH() public {
        vm.expectRevert(bytes("Different Words!")); //expect a revert to happen
        fundMe.fund();
    }

Result in Terminal:

equious@DESKTOP-HTA2B2C:~/github-discussion-help/Foundry__FundMe$ forge test --mt testFailsWithoutEnoughETH -vvvv
[⠒] Compiling...
[⠃] Compiling 3 files with 0.8.21
[⠊] Solc 0.8.21 finished in 662.23ms
Compiler run successful!

Running 1 test for test/FundMeTest.t.sol:FundMeTest
[PASS] testFailsWithoutEnoughETH() (gas: 8672)
Traces:
  [8672] FundMeTest::testFailsWithoutEnoughETH()
    ├─ [0] VM::expectRevert(Different Words!)
    │   └─ ← ()
    ├─ [263] FundMe::fund()
    │   └─ ← revert: You need to spend more ETH!
    └─ ← Error != expected error: You need to spend more ETH! != Different Words!

Test result: ok. 1 passed; 0 failed; 0 skipped; finished in 1.38ms
 
Ran 1 test suites: 1 tests passed, 0 failed, 0 skipped (1 total tests)

The test seems to recognize the error is not what's expected, but passes anyway.

A further, perhaps related behaviour I can't explain is that removing the arguments passed to vm.expectRevert() - which should result in a pass for any revert - actually fails, despite catching the revert.

Function

function testFailsWithoutEnoughETH() public {
        vm.expectRevert(); //expect a revert to happen
        fundMe.fund();
    }

Result in Terminal:

  [8302] FundMeTest::testFailsWithoutEnoughETH()
    ├─ [0] VM::expectRevert(custom error f4844814:)
    │   └─ ← ()
    ├─ [263] FundMe::fund()
    │   └─ ← revert: You need to spend more ETH!
    └─ ← ()

Test result: FAILED. 0 passed; 1 failed; 0 skipped; finished in 1.28ms
 
Ran 1 test suites: 0 tests passed, 1 failed, 0 skipped (1 total tests)

Failing tests:
Encountered 1 failing test in test/FundMeTest.t.sol:FundMeTest
[FAIL. Reason: assertion failed] testFailsWithoutEnoughETH() (gas: 8302)

Encountered a total of 1 failing tests, 0 tests succeeded

[DaniPopes]

What Foundry version are you on?

[Equious]

v0.2.0 @DaniPopes

[gakonst]

@Equious Can you share your forge -v? We haven't updated the version in the CLI but the date should update

[Equious]

Here you are! @gakonst

[DaniPopes]

Ah I see what's happening here, @Equious your test name starts with "testFail" which expects the test to fail. Either rename the test to test_fails... (or anything other than starting with exactly "testFail"), or remove the expectRevert