Skip to content

Commit

Permalink
Fix minimum_coverage_by_file check & prep 0.21.1 (#966)
Browse files Browse the repository at this point in the history
minimum_coverage_by_file was a gaping hole in the test suite,
which is now partially remedied through rather thorough cucumber
scenarios and 2 small unit tests that should be expanded on in
the future.

On the negative side: This will inconvenince some people. On the
positive side stuffed another hole in the test suite towards a
better future :D Also it was reported very soon thanks to @iMacTia

Fixes #965
  • Loading branch information
PragTob authored Jan 4, 2021
1 parent 20af434 commit 120d0ea
Show file tree
Hide file tree
Showing 8 changed files with 121 additions and 22 deletions.
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
0.21.1 (2021-01-04)
==========

## Bugfixes
* `minimum_coverage_by_file` works again as expected (errored out before 😱)

0.21.0 (2021-01-03)
==========

Expand Down
2 changes: 1 addition & 1 deletion Gemfile.lock
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
PATH
remote: .
specs:
simplecov (0.21.0)
simplecov (0.21.1)
docile (~> 1.1)
simplecov-html (~> 0.11)
simplecov_json_formatter (~> 0.1)
Expand Down
18 changes: 3 additions & 15 deletions features/minimum_coverage.feature
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Feature:
Background:
Given I'm working on the project "faked_project"

Scenario:
Scenario: It fails against too high coverage
Given SimpleCov for Test/Unit is configured with:
"""
require 'simplecov'
Expand All @@ -22,7 +22,7 @@ Feature:
And the output should contain "Line coverage (88.09%) is below the expected minimum coverage (90.00%)."
And the output should contain "SimpleCov failed with exit 2"

Scenario:
Scenario: It fails if it's just 0.01% too low
Given SimpleCov for Test/Unit is configured with:
"""
require 'simplecov'
Expand All @@ -37,7 +37,7 @@ Feature:
And the output should contain "Line coverage (88.09%) is below the expected minimum coverage (88.10%)."
And the output should contain "SimpleCov failed with exit 2"

Scenario:
Scenario: It passes when it is exactly the coverage
Given SimpleCov for Test/Unit is configured with:
"""
require 'simplecov'
Expand All @@ -50,18 +50,6 @@ Feature:
When I run `bundle exec rake test`
Then the exit status should be 0

Scenario:
Given SimpleCov for Test/Unit is configured with:
"""
require 'simplecov'
SimpleCov.start do
add_filter 'test.rb'
end
"""

When I run `bundle exec rake test`
Then the exit status should be 0

@branch_coverage
Scenario: Works together with branch coverage and the new criterion announcing both failures
Given SimpleCov for Test/Unit is configured with:
Expand Down
72 changes: 72 additions & 0 deletions features/minimum_coverage_by_file.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
@test_unit @config
Feature:

Exit code should be non-zero if the coverage of any one file is below the configured value.

Background:
Given I'm working on the project "faked_project"

Scenario: slightly under minimum coverage by file
Given SimpleCov for Test/Unit is configured with:
"""
require 'simplecov'
SimpleCov.start do
add_filter 'test.rb'
minimum_coverage_by_file 75.01
end
"""

When I run `bundle exec rake test`
Then the exit status should not be 0
And the output should contain "Line coverage by file (75.00%) is below the expected minimum coverage (75.01%)."
And the output should contain "SimpleCov failed with exit 2"

Scenario: Just passing it
Given SimpleCov for Test/Unit is configured with:
"""
require 'simplecov'
SimpleCov.start do
add_filter 'test.rb'
minimum_coverage_by_file 75
end
"""

When I run `bundle exec rake test`
Then the exit status should be 0

@branch_coverage
Scenario: Works together with branch coverage and the new criterion announcing both failures
Given SimpleCov for Test/Unit is configured with:
"""
require 'simplecov'
SimpleCov.start do
add_filter 'test.rb'
enable_coverage :branch
minimum_coverage_by_file line: 90, branch: 70
end
"""

When I run `bundle exec rake test`
Then the exit status should not be 0
And the output should contain "Line coverage by file (80.00%) is below the expected minimum coverage (90.00%)."
And the output should contain "Branch coverage by file (50.00%) is below the expected minimum coverage (70.00%)."
And the output should contain "SimpleCov failed with exit 2"

@branch_coverage
Scenario: Can set branch as primary coverage and it will fail if branch is below minimum coverage
Given SimpleCov for Test/Unit is configured with:
"""
require 'simplecov'
SimpleCov.start do
add_filter 'test.rb'
enable_coverage :branch
primary_coverage :branch
minimum_coverage_by_file 70
end
"""

When I run `bundle exec rake test`
Then the exit status should not be 0
And the output should contain "Branch coverage by file (50.00%) is below the expected minimum coverage (70.00%)."
And the output should not contain "Line coverage"
And the output should contain "SimpleCov failed with exit 2"
6 changes: 3 additions & 3 deletions lib/simplecov/exit_codes/minimum_coverage_by_file_check.rb
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ def failing?
def report
minimum_violations.each do |violation|
$stderr.printf(
"%<criterion>s coverage (%<covered>.2f%%) is below the expected minimum coverage (%<minimum_coverage>.2f%%).\n",
"%<criterion>s coverage by file (%<covered>.2f%%) is below the expected minimum coverage (%<minimum_coverage>.2f%%).\n",
covered: SimpleCov.round_coverage(violation.fetch(:actual)),
minimum_coverage: violation.fetch(:minimum_expected),
criterion: violation.fetch(:criterion).capitalize
Expand All @@ -40,11 +40,11 @@ def minimum_violations

def compute_minimum_coverage_data
minimum_coverage_by_file.flat_map do |criterion, expected_percent|
result.coverage_statistics_by_file[criterion].map do |actual_percent|
result.coverage_statistics_by_file.fetch(criterion).map do |actual_coverage|
{
criterion: criterion,
minimum_expected: expected_percent,
actual: SimpleCov.round_coverage(actual_percent)
actual: SimpleCov.round_coverage(actual_coverage.percent)
}
end
end
Expand Down
4 changes: 2 additions & 2 deletions lib/simplecov/file_list.rb
Original file line number Diff line number Diff line change
Expand Up @@ -106,8 +106,8 @@ def branch_covered_percent

def compute_coverage_statistics_by_file
@files.each_with_object(line: [], branch: []) do |file, together|
together[:line] << file.coverage_statistics[:line]
together[:branch] << file.coverage_statistics[:branch] if SimpleCov.branch_coverage?
together[:line] << file.coverage_statistics.fetch(:line)
together[:branch] << file.coverage_statistics.fetch(:branch) if SimpleCov.branch_coverage?
end
end

Expand Down
2 changes: 1 addition & 1 deletion lib/simplecov/version.rb
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# frozen_string_literal: true

module SimpleCov
VERSION = "0.21.0"
VERSION = "0.21.1"
end
33 changes: 33 additions & 0 deletions spec/exit_codes/minimum_coverage_by_file_check_spec.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# frozen_string_literal: true

require "helper"

RSpec.describe SimpleCov::ExitCodes::MinimumCoverageByFileCheck do
let(:result) do
instance_double(SimpleCov::Result, coverage_statistics_by_file: stats)
end
let(:stats) do
{
line: [SimpleCov::CoverageStatistics.new(covered: 8, missed: 2)]
}
end

subject { described_class.new(result, minimum_coverage_by_file) }

context "all files passing requirements" do
let(:minimum_coverage_by_file) { {line: 80} }

it "passes" do
expect(subject).not_to be_failing
end
end

context "one file violating requirements" do
let(:minimum_coverage_by_file) { {line: 90} }

it "fails" do
p minimum_coverage_by_file
expect(subject).to be_failing
end
end
end

0 comments on commit 120d0ea

Please sign in to comment.