Add a flaky result type to be used for flaky scenarios. #141
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* The flaky result type should be ok in non-strict mode and not ok in strict mode, so the exit code becomes zero with flaky scenarios in non-strict mode. * Update the Summary report to detect and count flaky scenarios.
This was referenced Jul 14, 2017
netbsd-srcmastr
pushed a commit
to NetBSD/pkgsrc
that referenced
this pull request
Mar 21, 2018
## [3.0.0](cucumber/cucumber-ruby-core@v3.0.0.pre.2...v3.0.0) (2017-09-27) ### Changed * Step#name renamed to #text ([#137](cucumber/cucumber-ruby-core#137) [@olleolleolle](https://github.com/olleolleolle)) * Use past tense in event names (`xStarting` -> `xStarted`) (see [cucumber/cucumber-ruby#1166](cucumber/cucumber-ruby#1166) @brasmusson). ### Added * Do not create test cases for scenarios with no steps ([#144](cucumber/cucumber-ruby-core#144) @brasmusson) * Handle selective strict settings ([#143](cucumber/cucumber-ruby-core#143) @brasmusson) ### Fixed * Fix DataTable's Location to be aware of all of its lines ([#142](cucumber/cucumber-ruby-core#142) @botandrose) ### Improved * As per [#251](cucumber/common#251): renamed History.md to CHANGELOG.md, added contributing message at beginning, and misc formatting. ([#145](cucumber/cucumber-ruby-core#145) [jaysonesmith](https://github.com/jaysonesmith)) ## [3.0.0.pre.2](cucumber/cucumber-ruby-core@v2.0.0...3.0.0.pre.2) (2017-07-26) ### New Features * Add a flaky result type to be used for flaky scenarios ([#141](cucumber/cucumber-ruby-core#141), [cucumber/cucumber-ruby#1044](cucumber/cucumber-ruby#1044) @brasmusson) * Make the Summary report able to say if the total result is ok ([#140](cucumber/cucumber-ruby-core#140) @brasmusson) * Replay previous events to new subscribers ([#136](cucumber/cucumber-ruby-core#136) @mattwynne) * Ruby 2.4.0 compatibility ([#120](cucumber/cucumber-ruby-core#120) @junaruga) * Use tag expressions ([#116](cucumber/cucumber-ruby-core#116) @brasmusson) * Access example table row data by param name ([#118](cucumber/cucumber-ruby-core#118) @enkessler) ### Bugfixes N/A ### Removed Features N/A ### Refactoring * Travis: jruby-9.1.10.0 ([#130](cucumber/cucumber-ruby-core#130) @olleolleolle) * Travis: jruby-9.1.12.0 ([#133](cucumber/cucumber-ruby-core#132) @olleolleolle)
|
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Add a flaky result type to be used for flaky scenarios.
Details
Add a flaky result type to be used for flaky scenarios.
strict mode, so the exit code becomes zero with flaky scenarios in
non-strict mode.
Fixes cucumber/cucumber-ruby#1044.
Motivation and Context
After the
--retryfeature was introduced it makes sense not to report each execution of the scenarios which are retried in the summary, but each retried scenario once - either asfailedorflakydepending on if they passes after retry or not.How Has This Been Tested?
The automated test suite is updated to verify this new behaviour.
Types of changes
Checklist: