Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for checks_to_ignore #151

Merged
merged 4 commits into from
Jan 29, 2015
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,12 +56,12 @@ Project | Repository

### Favicon

Checks if your favicons are valid. This is an optional feature, set the `validate_favicon` option to turn it on.
Checks if your favicons are valid. This is an optional feature, set the `check_favicon` option to turn it on.

### HTML

Nokogiri looks at the markup and [provides errors](http://www.nokogiri.org/tutorials/ensuring_well_formed_markup.html) when parsing your document.
This is an optional feature, set the `validate_html` option to enable validation errors from Nokogiri.
This is an optional feature, set the `check_html` option to enable validation errors from Nokogiri.

## Usage

Expand Down Expand Up @@ -139,6 +139,7 @@ The `HTML::Proofer` constructor takes an optional hash of additional options:
| :----- | :---------- | :------ |
| `alt_ignore` | An array of Strings or RegExps containing `img`s whose missing `alt` tags are safe to ignore. | `[]` |
| `check_external_hash` | Checks whether external hashes exist (even if the website exists). This slows the checker down. | `false` |
|`checks_to_ignore`| An array of Strings indicating which checks you'd like to not perform. | `[]`
| `directory_index_file` | Sets the file to look for when a link refers to a directory. | `index.html` |
| `disable_external` | If `true`, does not run the external link checker, which can take a lot of time. | `false` |
| `error_sort` | Defines the sort order for error output. Can be `:path`, `:desc`, or `:status`. | `:path`
Expand All @@ -147,8 +148,8 @@ The `HTML::Proofer` constructor takes an optional hash of additional options:
| `href_ignore` | An array of Strings or RegExps containing `href`s that are safe to ignore. Note that non-HTTP(S) URIs are always ignored. | `[]` |
| `href_swap` | A hash containing key-value pairs of `RegExp => String`. It transforms links that match `RegExp` into `String` via `gsub`. | `{}` |
| `only_4xx` | Only reports errors for links that fall within the 4xx status code range. | `false` |
| `validate_favicon` | Enables the favicon checker. | `false` |
| `validate_html` | Enables HTML validation errors from Nokogiri | `false` |
| `check_favicon` | Enables the favicon checker. | `false` |
| `check_html` | Enables HTML validation errors from Nokogiri | `false` |
| `verbose` | If `true`, outputs extra information as the checking happens. Useful for debugging. | `false` |

### Configuring Typhoeus and Hydra
Expand Down Expand Up @@ -200,9 +201,9 @@ Add the `data-proofer-ignore` attribute to any tag to ignore it from the checks.

## Custom tests

Want to write your own test? Sure! Just create two classes--one that inherits from `HTML::Proofer::Runner`, and another that inherits from `HTML::Proofer::Checkable`.
Want to write your own test? Sure! Just create two classes--one that inherits from `HTML::Proofer::CheckRunner`, and another that inherits from `HTML::Proofer::Checkable`.

The `Runner` subclass must define one method called `run`. This is called on your content, and is responsible for performing the validation on whatever elements you like. When you catch a broken issue, call `add_issue(message)` to explain the error.
The `CheckRunner` subclass must define one method called `run`. This is called on your content, and is responsible for performing the validation on whatever elements you like. When you catch a broken issue, call `add_issue(message)` to explain the error.

The `Checkable` subclass defines various helper methods you can use as part of your test. Usually, you'll want to instantiate it within `run`. You have access to all of your element's attributes.

Expand All @@ -222,7 +223,7 @@ class OctocatLinkCheck < ::HTML::Proofer::Checkable

end

class MailToOctocat < ::HTML::Proofer::Runner
class MailToOctocat < ::HTML::Proofer::CheckRunner

def run
@html.css('a').each do |l|
Expand Down
5 changes: 3 additions & 2 deletions bin/htmlproof
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ Mercenary.program(:htmlproof) do |p|

p.option 'as_links', '--as-links', 'Assumes that `PATH` is a comma-separated array of links to check.'
p.option 'alt_ignore', '--alt-ignore image1,[image2,...]', Array, 'Comma-separated list of Strings or RegExps containing `img`s whose missing `alt` tags are safe to ignore'
p.option 'checks_to_ignore', '--checks-to-ignore check1,[check2,...]', Array, ' An array of Strings indicating which checks you\'d like to not perform.'
p.option 'check_external_hash', '--check-external-hash', 'Checks whether external hashes exist (even if the website exists). This slows the checker down (default: `false`).'
p.option 'directory_index_file', '--directory-index-file', String, 'Sets the file to look for when a link refers to a directory. (default: `index.html`)'
p.option 'disable_external', '--disable-external', 'Disables the external link checker (default: `false`)'
Expand All @@ -33,8 +34,8 @@ Mercenary.program(:htmlproof) do |p|
p.option 'href_ignore', '--href-ignore link1,[link2,...]', Array, 'Comma-separated list of Strings or RegExps containing `href`s that are safe to ignore.'
p.option 'href_swap', '--href-swap re:string,[re:string,...]', Array, 'Comma-separated list of key-value pairs of `RegExp:String`. Transforms links matching `RegExp` into `String`'
p.option 'only_4xx', '--only-4xx', 'Only reports errors for links that fall within the 4x status code range.'
p.option 'validate_favicon', '--validate-favicon', 'Enables the favicon checker (default: `false`).'
p.option 'validate_html', '--validate-html', 'Enables HTML validation errors from Nokogiri (default: `false`).'
p.option 'check_favicon', '--check-favicon', 'Enables the favicon checker (default: `false`).'
p.option 'check_html', '--check-html', 'Enables HTML validation errors from Nokogiri (default: `false`).'
p.option 'verbose', '--verbose', 'Enables more verbose logging.'

p.action do |args, opts|
Expand Down
23 changes: 14 additions & 9 deletions lib/html/proofer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ def require_all(path)
end

require_all 'proofer'
require_all 'proofer/runner'
require_all 'proofer/check_runner'
require_all 'proofer/checks'

require 'parallel'
Expand All @@ -27,7 +27,7 @@ def initialize(src, opts = {})

@proofer_opts = {
:ext => '.html',
:validate_favicon => false,
:check_favicon => false,
:href_swap => [],
:href_ignore => [],
:file_ignore => [],
Expand All @@ -37,8 +37,9 @@ def initialize(src, opts = {})
:verbose => false,
:only_4xx => false,
:directory_index_file => 'index.html',
:validate_html => false,
:error_sort => :path
:check_html => false,
:error_sort => :path,
:checks_to_ignore => []
}

@typhoeus_opts = opts[:typhoeus] || {
Expand Down Expand Up @@ -147,10 +148,14 @@ def ignore_file?(file)
end

def checks
checks = HTML::Proofer::Runner.checks.map(&:name)
checks.delete('FaviconRunner') unless @options[:validate_favicon]
checks.delete('HtmlRunner') unless @options[:validate_html]
checks
return @checks unless @checks.nil?
@checks = HTML::Proofer::CheckRunner.checks.map(&:name)
@checks.delete('FaviconCheck') unless @options[:check_favicon]
@checks.delete('HtmlCheck') unless @options[:check_html]
@options[:checks_to_ignore].each do |ignored|
@checks.delete(ignored)
end
@checks
end

def failed_tests
Expand All @@ -161,7 +166,7 @@ def failed_tests
end

def print_failed_tests
sorted_failures = HTML::Proofer::Runner::SortedIssues.new(@failed_tests, @options[:error_sort], logger)
sorted_failures = HTML::Proofer::CheckRunner::SortedIssues.new(@failed_tests, @options[:error_sort], logger)

sorted_failures.sort_and_report
count = @failed_tests.length
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
class HTML::Proofer

# Mostly handles issue management and collecting of external URLs.
class Runner
class CheckRunner

attr_reader :issues, :src, :path, :options, :external_urls, :href_ignores, :alt_ignores

Expand All @@ -19,7 +19,7 @@ def initialize(src, path, html, opts={})
end

def run
fail NotImplementedError, 'HTML::Proofer::Runner subclasses must implement #run'
fail NotImplementedError, 'HTML::Proofer::CheckRunner subclasses must implement #run'
end

def add_issue(desc, status = -1)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# encoding: utf-8
class HTML::Proofer::Runner
class HTML::Proofer::CheckRunner

class Issue
attr_reader :path, :desc, :status
Expand Down
6 changes: 3 additions & 3 deletions lib/html/proofer/checkable.rb
Original file line number Diff line number Diff line change
Expand Up @@ -66,11 +66,11 @@ def ignore?
return true if @data_ignore_proofer

case @type
when 'FaviconCheck'
when 'FaviconCheckable'
return true if url.match(/^data:image/)
when 'LinkCheck'
when 'LinkCheckable'
return true if ignores_pattern_check(@check.href_ignores)
when 'ImageCheck'
when 'ImageCheckable'
return true if url.match(/^data:image/)
return true if ignores_pattern_check(@check.alt_ignores)
end
Expand Down
6 changes: 3 additions & 3 deletions lib/html/proofer/checks/favicon.rb
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
# encoding: utf-8

class FaviconCheck < ::HTML::Proofer::Checkable
class FaviconCheckable < ::HTML::Proofer::Checkable
def rel
@rel
end
end

class FaviconRunner < ::HTML::Proofer::Runner
class FaviconCheck < ::HTML::Proofer::CheckRunner

def run
@html.xpath('//link[not(ancestor::pre or ancestor::code)]').each do |favicon|
favicon = FaviconCheck.new favicon, self
favicon = FaviconCheckable.new favicon, self
next if favicon.ignore?
return if favicon.rel.split(' ').last.eql? 'icon'
end
Expand Down
2 changes: 1 addition & 1 deletion lib/html/proofer/checks/html.rb
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# encoding: utf-8

class HtmlRunner < ::HTML::Proofer::Runner
class HtmlCheck < ::HTML::Proofer::CheckRunner

# new html5 tags (source: http://www.w3schools.com/html/html5_new_elements.asp)
HTML5_TAGS = %w(article aside bdi details dialog figcaption
Expand Down
6 changes: 3 additions & 3 deletions lib/html/proofer/checks/images.rb
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# encoding: utf-8

class ImageCheck < ::HTML::Proofer::Checkable
class ImageCheckable < ::HTML::Proofer::Checkable

SCREEN_SHOT_REGEX = /Screen(?: |%20)Shot(?: |%20)\d+-\d+-\d+(?: |%20)at(?: |%20)\d+.\d+.\d+/

Expand All @@ -22,10 +22,10 @@ def missing_src?

end

class ImageRunner < ::HTML::Proofer::Runner
class ImageCheck < ::HTML::Proofer::CheckRunner
def run
@html.css('img').each do |i|
img = ImageCheck.new i, self
img = ImageCheckable.new i, self

next if img.ignore?

Expand Down
6 changes: 3 additions & 3 deletions lib/html/proofer/checks/links.rb
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# encoding: utf-8
require_relative '../utils'

class LinkCheck < ::HTML::Proofer::Checkable
class LinkCheckable < ::HTML::Proofer::Checkable

def href
real_attr @href
Expand All @@ -25,12 +25,12 @@ def placeholder?

end

class LinkRunner < ::HTML::Proofer::Runner
class LinkCheck < ::HTML::Proofer::CheckRunner
include HTML::Utils

def run
@html.css('a, link').each do |l|
link = LinkCheck.new l, self
link = LinkCheckable.new l, self

next if link.ignore?
next if link.href =~ /^javascript:/ # can't put this in ignore? because the URI does not parse
Expand Down
6 changes: 3 additions & 3 deletions lib/html/proofer/checks/scripts.rb
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# encoding: utf-8

class ScriptCheck < ::HTML::Proofer::Checkable
class ScriptCheckable < ::HTML::Proofer::Checkable

def src
real_attr @src
Expand All @@ -16,10 +16,10 @@ def blank?

end

class ScriptRunner < ::HTML::Proofer::Runner
class ScriptCheck < ::HTML::Proofer::CheckRunner
def run
@html.css('script').each do |s|
script = ScriptCheck.new s, self
script = ScriptCheckable.new s, self

next if script.ignore?
next unless script.blank?
Expand Down
4 changes: 2 additions & 2 deletions lib/html/proofer/url_validator.rb
Original file line number Diff line number Diff line change
Expand Up @@ -116,9 +116,9 @@ def handle_timeout

def add_failed_tests(filenames, desc, status = nil)
if filenames.nil?
@failed_tests << Runner::Issue.new('', desc, status)
@failed_tests << CheckRunner::Issue.new('', desc, status)
else
filenames.each { |f| @failed_tests << Runner::Issue.new(f, desc, status) }
filenames.each { |f| @failed_tests << CheckRunner::Issue.new(f, desc, status) }
end
end

Expand Down
16 changes: 12 additions & 4 deletions spec/html/proofer/command_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,13 @@
expect(output).to match('successfully')
end

it 'works with checks_to_ignore' do
external = "#{FIXTURES_DIR}/links/file.foo"
output = make_bin('--ext .foo --checks-to-ignore LinkCheck', external)
expect(output).to match('successfully')
expect(output).to_not match('LinkCheck')
end

it 'works with check-external-hash' do
brokenHashOnTheWeb = "#{FIXTURES_DIR}/links/brokenHashOnTheWeb.html"
output = make_bin('--check-external-hash', brokenHashOnTheWeb)
Expand All @@ -34,6 +41,7 @@
external = "#{FIXTURES_DIR}/links/file.foo"
output = make_bin('--ext .foo', external)
expect(output).to match('1 failure')
expect(output).to match('LinkCheck')
end

it 'works with file-ignore' do
Expand All @@ -60,15 +68,15 @@
expect(output).to match('successfully')
end

it 'works with validate-favicon' do
it 'works with check-favicon' do
broken = "#{FIXTURES_DIR}/favicon/favicon_broken.html"
output = make_bin('--validate-favicon', broken)
output = make_bin('--check-favicon', broken)
expect(output).to match('1 failure')
end

it 'works with validate-html' do
it 'works with check-html' do
broken = "#{FIXTURES_DIR}/html/invalid_tag.html"
output = make_bin('--validate-html', broken)
output = make_bin('--check-html', broken)
expect(output).to match('1 failure')
end
end
14 changes: 7 additions & 7 deletions spec/html/proofer/favicon_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,44 +3,44 @@
describe 'Favicons test' do
it 'ignores for absent favicon by default' do
absent = "#{FIXTURES_DIR}/favicon/favicon_absent.html"
expect(make_proofer(absent).failed_tests).to eq []
expect(run_proofer(absent).failed_tests).to eq []
end

it 'fails for absent favicon' do
absent = "#{FIXTURES_DIR}/favicon/favicon_absent.html"
proofer = make_proofer(absent, { :validate_favicon => true })
proofer = run_proofer(absent, { :check_favicon => true })
expect(proofer.failed_tests.first).to match(/no favicon specified/)
end

it 'fails for absent favicon but present apple touch icon' do
absent = "#{FIXTURES_DIR}/favicon/favicon_absent_apple.html"
proofer = make_proofer(absent, { :validate_favicon => true })
proofer = run_proofer(absent, { :check_favicon => true })
# Travis gives a different error message here for some reason
expect(proofer.failed_tests.last).to match(/(internally linking to gpl.png, which does not exist|no favicon specified)/)
end

it 'fails for broken favicon' do
broken = "#{FIXTURES_DIR}/favicon/favicon_broken.html"
proofer = make_proofer(broken, { :validate_favicon => true })
proofer = run_proofer(broken, { :check_favicon => true })

expect(proofer.failed_tests.first).to match(/internally linking to asdadaskdalsdk.png/)
end

it 'passes for present favicon' do
present = "#{FIXTURES_DIR}/favicon/favicon_present.html"
proofer = make_proofer(present, { :validate_favicon => true })
proofer = run_proofer(present, { :check_favicon => true })
expect(proofer.failed_tests).to eq []
end

it 'passes for present favicon with shortcut notation' do
present = "#{FIXTURES_DIR}/favicon/favicon_present_shortcut.html"
proofer = make_proofer(present, { :validate_favicon => true })
proofer = run_proofer(present, { :check_favicon => true })
expect(proofer.failed_tests).to eq []
end

it 'fails for broken favicon with data-proofer-ignore' do
broken_but_ignored = "#{FIXTURES_DIR}/favicon/favicon_broken_but_ignored.html"
proofer = make_proofer(broken_but_ignored, { :validate_favicon => true })
proofer = run_proofer(broken_but_ignored, { :check_favicon => true })
expect(proofer.failed_tests.first).to match(/no favicon specified/)
end

Expand Down
Loading