A more minimal approach to retrying failed examples #596
Conversation
Intended for use with -O; outputs "-e" followed by the full name of the failed example.
A raw split causes carefully-escaped example names to get screwed up. It's also more consistent with the spirit of the external options files (equivalence with what you might provide on the command line).
It will now run examples whose names match any of the provided regular expressions.
Ties together the failure formatter, the additive -e, and use of the supplemental -O command-line options file.
|
I've pulled two parts from here into separate pull requests.
|
|
I think maybe the formatter should output into a well known file though, like |
|
Not sure I follow the last question. I think it works just like the other formatters, where a pair of The feature in 835f24d almost does this. I think if I'd left off the |
|
Maybe I am missing something, but |
Split the feature to hopefully assuage dblock's concern about how multiple output/format specifications work.
|
While this implementation is being debated, we pulled #610 and #614 and used this idea (and roughly this code) to do this outside of RSpec proper. Blogged http://artsy.github.com/blog/2012/05/15/how-to-organize-over-3000-rspec-specs-and-retry-test-failures/. |
|
Thanks for that blog. I think I'll probably just leave well enough alone and do it outside of rspec as well. You're doing something slightly fancier than what we are, but don't mind if I borrow some inspiration from your example! :-) |
|
FWIW, I've pushed another solution -- a gem which wraps the rspec command which does not require code changes to the project under test -- here: http://github.com/oggy/respec . Will make this use -e once pull #614 lands in stable. |
|
I'd be in favor of including this formatter in core. Users coming from cucumber are going to look for and expect an equivalent to cucumber's Now that #610 and #614 have been merged into master, it looks like the only thing still needed for this feature to be available is the Any chance of seeing this final small piece merged in as well? Otherwise, we should extract I don't really like the idea of using a completely separate @dblock's blog post was valuable in showing how to pull everything together using rake tasks (and makes me think we should suggest adding a |
|
Here we go: https://github.com/dblock/rspec-rerun The formatter is included, with specs, I'm working on a README and stuff like that. It needs built-in rake tasks to do retry s well, please contribute. |
|
... and it turned out to be simpler than I thought. Released rspec-rerun 0.1.0. To use, just add it to Gemfile and change the default task in Rakefile: require 'rspec-rerun'
task :default => "rspec-rerun:spec" |
|
Check out #456 (comment) for another option. |
So after the discussion on #456, here's a crack at a less-invasive way to do the retry.
Given a formatter that outputs the names of failed examples only like so:
You can get that output into a file using an rspec invocation like so:
And then upon determining that some examples have failed, rerun just those examples like so:
In order to make this possible I had to make two changes to rspec itself, not counting the formatter: -e is now additive, and options provided in the -O file (and in .rspec, etc.) are now split using shell semantics rather than on whitespace (which I would have considered a bug).
Let me know what you think. The formatter is perhaps controversial if only because it's so trivial. I favor just going ahead and making it work out of the box, but I could also be convinced of alternatives like splitting it out into a separate distribution or having it not output the "-e" hiccup and leaving the prepending of "-e" to each line in this usage to the enterprising rspec user.