Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow warnings to be hidden #309

Merged
merged 9 commits into from
Dec 20, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions features/hiding_errors.feature
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Feature: Hiding errors

Things like ORB and Barlesque that we don't control will clutter up our pages
with violations.
Websites often contain external parts that we don't control, which will
clutter up our pages with violations.

We need a way to ignore these kinds of problems, so we can focus on the
problems that we can actually fix.
Expand Down
39 changes: 39 additions & 0 deletions features/hiding_warnings.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
Feature: Hiding warnings

Websites often contain external parts that we don't control, which will
clutter up our pages with violations.

We need a way to ignore these kinds of problems, so we can focus on the
problems that we can actually fix.

Scenario: Hide warnings matching a selector
Given a website running at http://localhost:54321
And a file named "a11y.js" with:
"""
page("http://localhost:54321/one_warning.html", {
hide: "@id='main'"
})
"""
When I run `bbc-a11y`
Then it should pass with:
"""
✓ http://localhost:54321/one_warning.html

1 page checked, 0 errors found, 0 warnings, 1 warning hidden, 0 standards skipped
"""

Scenario: Hide warnings matching a rule
Given a website running at http://localhost:54321
And a file named "a11y.js" with:
"""
page("http://localhost:54321/one_warning.html", {
hide: "First heading was not a main heading"
})
"""
When I run `bbc-a11y`
Then it should pass with:
"""
✓ http://localhost:54321/one_warning.html

1 page checked, 0 errors found, 0 warnings, 1 warning hidden, 0 standards skipped
"""
13 changes: 13 additions & 0 deletions features/support/web_server/one_warning.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!DOCTYPE html>
<html lang="en-gb">
<head>
<title>Blank</title>
</head>
<body>
<div id="main" role="main">
<h2>h2</h2>
<h1>h1</h1>
<p>hello</p>
</div>
</body>
</html>
3 changes: 3 additions & 0 deletions lib/reporters/pretty.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ PrettyReporter.prototype.runStarted = function () {
errorsFound: 0,
errorsHidden: 0,
warningsFound: 0,
warningsHidden: 0,
standardsSkipped: 0,
documentationUrls: []
}
Expand Down Expand Up @@ -68,6 +69,7 @@ PrettyReporter.prototype.pageChecked = function (page, pageResult) {
pageResult.results.forEach(function (standardResult) {
summary.errorsHidden += standardResult.hiddenErrors.length
summary.warningsFound += standardResult.warnings.length
summary.warningsHidden += standardResult.hiddenWarnings.length
})

var hasErrors = pageResult.results.some(r => r.errors.length > 0)
Expand Down Expand Up @@ -116,6 +118,7 @@ PrettyReporter.prototype.runEnded = function () {
pluralise(this.summary.errorsFound, 'error') + ' found, ' +
(this.summary.errorsHidden > 0 ? pluralise(this.summary.errorsHidden, 'error') + ' hidden, ' : '') +
pluralise(this.summary.warningsFound, 'warning') + ', ' +
(this.summary.warningsHidden > 0 ? pluralise(this.summary.warningsHidden, 'warning') + ' hidden, ' : '') +
pluralise(this.summary.standardsSkipped, 'standard') + ' skipped'
)
this.commandLineConsole.log('')
Expand Down
15 changes: 15 additions & 0 deletions lib/results/standardResult.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ function StandardResult (section, name, hide) {
this.errors = []
this.warnings = []
this.hiddenErrors = []
this.hiddenWarnings = []
this.hide = hide
}

Expand All @@ -28,6 +29,20 @@ StandardResult.prototype.addError = function (errorChain) {

StandardResult.prototype.addWarning = function (errorChain) {
this.warnings.push(xpath.replaceElementsWithXPaths(errorChain))

if (this.hide) {
var standardResult = this
var hide = this.hide
standardResult.hiddenWarnings = standardResult.warnings.filter(function (warning) {
var fullWarningName = warning.map(function (e) { return e.xpath || e }).join(' ')
return (hide.find(function (substring) {
return fullWarningName.indexOf(substring) > -1
}))
})
standardResult.warnings = standardResult.warnings.filter(function (warning) {
return standardResult.hiddenWarnings.indexOf(warning) === -1
})
}
}

StandardResult.prototype.flatten = function () {
Expand Down
12 changes: 12 additions & 0 deletions test/a11ySpec.js
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,18 @@ describe('a11y', function () {
})
})

it('hides warnings', function () {
$('body').append('<h2>I am in the wrong place</h2><h1>Hello!</h1>')
return a11y.test({ hide: ['First heading was not a main heading'] })
.then(function (outcome) {
var resultsWithWarnings = outcome.results.filter(function (standardResult) {
return standardResult.hiddenWarnings.length > 0
})
expect(resultsWithWarnings[0].warnings).to.eql([])
expect(resultsWithWarnings[0].hiddenWarnings[0][0]).to.eql('First heading was not a main heading:')
})
})

it('only runs specific standards', function () {
return a11y.test({ only: ['Structure: Containers and landmarks: Exactly one main landmark'] })
.then(function (outcome) {
Expand Down