`should_not raise_error` annoyingly swallows original exception backtrace #59

myronmarston · 2011-02-07T19:35:44Z

Given a spec like this:

describe "something" do
  it "does not raise an error" do
    expect { something }.to_not raise_error
  end
end

When an error is raised, the spec fails (as it should) but the failure backtrace isn't very useful. The backtrace I care about is for the original error, not the error raised by the raise_error matcher. Really, it would be better to use it("does not raise an error") { something } but the matcher may still be useful in some situations (i.e. to ensure every spec has at least one expectation) and I've got some error reports that have unhelpful backtraces because of this (see this VCR issue, for example).

It would be nice to have the raise_error matcher preserve the original exception backtrace. A few ideas:

Maybe raise_error in the should_not case should be pure syntactic sugar that doesn't actually do anything but execute the block? That way the original error (with associated backtrace) will bubble up and get printed. Users can still use the matcher to make it clear what expectation they are setting for the example.
Alternately, the matcher could print out the original exception's backtrace as part of the failure message. Presumably, we'd want it to respect the full_backtrace configuration option, although we'd need to find a way to do that in such a way that it doesn't couple rspec-expectations to rspec-core.

Myron

The text was updated successfully, but these errors were encountered:

dchelimsky · 2011-02-18T19:06:31Z

I'd prefer option 2.

alindeman · 2011-04-16T01:55:03Z

What are the pros of 2? To me, 1 sounds better.

dchelimsky · 2011-04-16T09:52:21Z

It's a failed expectation. I think it's more consistent with other failure messages if this gets wrapped up in a failure message rather than bubbling up directly. Why do you like 1 better?

chendo · 2011-12-20T06:26:29Z

I haven't got the time to write proper tests etc for it at the moment, but this is a fairly simple version of the RaiseException matcher, but puts the backtrace the message. Probably could do with cleaning out the backtrace etc, but it works (at least with 1.3.2)

https://gist.github.com/1500276

This is in prep for #59--once I had some backtrace info, all of these would fail since they are doing exact string matches.

myronmarston · 2012-09-11T05:27:34Z

I took a first pass at implementing number 2 from above, but it's tricky to figure out how to format this correctly. Here's what I've got so far:

  1) should_not raise_error includes the backtrace of the error that was raised in the error message
     Failure/Error: expect { raise "boom" }.not_to raise_error
       expected no Exception, got #<RuntimeError: boom> with backtrace:
         # /Users/myron/code/rspec-dev/repos/rspec-expectations/spec/rspec/matchers/raise_error_spec.rb:56:in `block (3 levels) in <top (required)>'
         # /Users/myron/code/rspec-dev/repos/rspec-expectations/lib/rspec/matchers/built_in/raise_error.rb:24:in `call'
         # /Users/myron/code/rspec-dev/repos/rspec-expectations/lib/rspec/matchers/built_in/raise_error.rb:24:in `matches?'
         # /Users/myron/code/rspec-dev/repos/rspec-expectations/lib/rspec/matchers/built_in/raise_error.rb:43:in `does_not_match?'
         # /Users/myron/code/rspec-dev/repos/rspec-expectations/lib/rspec/expectations/handler.rb:31:in `handle_matcher'
         # /Users/myron/code/rspec-dev/repos/rspec-expectations/lib/rspec/expectations/expectation_target.rb:48:in `to_not'
         # /Users/myron/code/rspec-dev/repos/rspec-expectations/spec/rspec/matchers/raise_error_spec.rb:56:in `block (2 levels) in <top (required)>'
         # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/example.rb:113:in `instance_eval'
         # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/example.rb:113:in `block in run'
         # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/example.rb:253:in `with_around_each_hooks'
         # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/example.rb:110:in `run'
         # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/example_group.rb:380:in `block in run_examples'
         # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/example_group.rb:376:in `map'
         # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/example_group.rb:376:in `run_examples'
         # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/example_group.rb:361:in `run'
         # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:28:in `block (2 levels) in run'
         # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:28:in `map'
         # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:28:in `block in run'
         # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:34:in `report'
         # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:25:in `run'
         # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:69:in `run'
         # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:8:in `block in autorun'
     # ./lib/rspec/expectations/fail_with.rb:33:in `fail_with'
     # ./lib/rspec/expectations/handler.rb:42:in `handle_matcher'
     # ./lib/rspec/expectations/expectation_target.rb:48:in `to_not'
     # ./spec/rspec/matchers/raise_error_spec.rb:56:in `block (2 levels) in <top (required)>'
     # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/example.rb:113:in `instance_eval'
     # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/example.rb:113:in `block in run'
     # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/example.rb:253:in `with_around_each_hooks'
     # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/example.rb:110:in `run'
     # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/example_group.rb:380:in `block in run_examples'
     # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/example_group.rb:376:in `map'
     # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/example_group.rb:376:in `run_examples'
     # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/example_group.rb:361:in `run'
     # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:28:in `block (2 levels) in run'
     # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:28:in `map'
     # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:28:in `block in run'
     # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:34:in `report'
     # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:25:in `run'
     # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:69:in `run'
     # /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:8:in `block in autorun'

I don't think this is a very promising route to take. Thoughts?

patmaddox · 2012-09-11T08:35:53Z

You want to go with #2 so that it is consistent with other expectations, formats nicely in the various reporters, etc

I think the formatting you've got there is fine. Perhaps some sort of dividing text between the error message and the failure message? I'm thinking of the spacing getting mangled on people's terminals and it being difficult to decipher exception message vs expectation failure.

myronmarston · 2012-09-11T14:32:11Z

The formatting consistency is a good point.

Anyhow, one other option I just thought of: looking at the above, it's clear that the two backtraces are nearly identical except for a few frames. We could set the backtrace on the expectation failure error to the backtrace of the rescued error, so that the user gets the full backtrace as the "main" backtrace while keeping a consistently formatted failure message.

Edit: another benefit of this approach is that I don't have to go through gymnastics to filter the backtrace the same as core does.

patmaddox · 2012-09-12T02:04:05Z

So just have it be one long backtrace, originating from the rescued error?

myronmarston · 2012-09-12T03:01:08Z

Yep. The message would be in the existing format, though.

Sent from my iPhone

On Sep 11, 2012, at 7:04 PM, Pat Maddox notifications@github.com wrote:

So just have it be one long backtrace, originating from the rescued error?

—
Reply to this email directly or view it on GitHub.

nyarly · 2012-10-02T19:07:02Z

I've wanted this for so long, it's painful. I think the output in the example is fine.

One important variation, though is that the positive matcher should work the same way: expect{ raise "boom"}.to raise_error(ArgumentError) should print the backtrace of the RuntimeError that actually gets raised.

@myronmarston - are you onto a PR already?

myronmarston · 2012-10-03T06:04:26Z

@myronmarston - are you onto a PR already?

Not yet. I've got the code lying around on my machine. There's some refactoring that needs to happen in rspec-core first to make it possible for this to use the same backtrace filtering logic core uses. Thanks for reminding me about this!

This will be used by rspec/rspec-expectations#59.

It's hard to troubleshoot unexpected errors when the backtrace is silenced, as it was previously. Closes #59.

myronmarston added a commit that referenced this issue Jul 13, 2012

Loosen error message constraints for raise_error matcher specs.

2c2fdb6

This is in prep for #59--once I had some backtrace info, all of these would fail since they are doing exact string matches.

myronmarston added a commit to rspec/rspec-core that referenced this issue Oct 5, 2012

Extract backtrace filtering methods into helper module.

b30c6ce

This will be used by rspec/rspec-expectations#59.

myronmarston closed this as completed Oct 5, 2012

This was referenced Oct 5, 2012

Include backtrace in errors reported by raise_error matcher. #177

Merged

Extract backtrace filtering methods into helper module. rspec/rspec-core#701

Merged

myronmarston added a commit to rspec/rspec-core that referenced this issue Oct 9, 2012

Extract BacktraceFormatters into a module.

39857f3

This will be used by rspec/rspec-expectations#59.

myronmarston added a commit that referenced this issue Oct 9, 2012

Include backtrace in errors reported by raise_error matcher.

4a919ab

It's hard to troubleshoot unexpected errors when the backtrace is silenced, as it was previously. Closes #59.

mdesantis mentioned this issue Oct 12, 2012

should_not raise_error annoyingly swallows original exception backtrace #179

Closed

myronmarston mentioned this issue Dec 29, 2012

Show warning for examples that don't include an expectation rspec/rspec-core#759

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`should_not raise_error` annoyingly swallows original exception backtrace #59

`should_not raise_error` annoyingly swallows original exception backtrace #59

myronmarston commented Feb 7, 2011

dchelimsky commented Feb 18, 2011

alindeman commented Apr 16, 2011

dchelimsky commented Apr 16, 2011

chendo commented Dec 20, 2011

myronmarston commented Sep 11, 2012

patmaddox commented Sep 11, 2012

myronmarston commented Sep 11, 2012

patmaddox commented Sep 12, 2012

myronmarston commented Sep 12, 2012

nyarly commented Oct 2, 2012

myronmarston commented Oct 3, 2012

should_not raise_error annoyingly swallows original exception backtrace #59

should_not raise_error annoyingly swallows original exception backtrace #59

Comments

myronmarston commented Feb 7, 2011

dchelimsky commented Feb 18, 2011

alindeman commented Apr 16, 2011

dchelimsky commented Apr 16, 2011

chendo commented Dec 20, 2011

myronmarston commented Sep 11, 2012

patmaddox commented Sep 11, 2012

myronmarston commented Sep 11, 2012

patmaddox commented Sep 12, 2012

myronmarston commented Sep 12, 2012

nyarly commented Oct 2, 2012

myronmarston commented Oct 3, 2012

`should_not raise_error` annoyingly swallows original exception backtrace #59

`should_not raise_error` annoyingly swallows original exception backtrace #59