Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor html-proofer #147

Merged
merged 40 commits into from
Jan 28, 2015
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
1c08ed4
Generic Rubocop fixups
gjtorikian Jan 1, 2015
995b2fb
%r{ appears to fail for Travis
gjtorikian Jan 1, 2015
d7e0606
Clean up the main entry class
gjtorikian Jan 19, 2015
286e2da
Move logging into its own class
gjtorikian Jan 24, 2015
a0a2a65
Improve some documentation
gjtorikian Jan 24, 2015
7b71cd7
Move UrlValidation out into its own class
gjtorikian Jan 24, 2015
efbb964
Obsolete
gjtorikian Jan 24, 2015
904811d
Include Utils here, too
gjtorikian Jan 24, 2015
b0b4102
Refactor all the check logic
gjtorikian Jan 24, 2015
c0deedf
Suffix all classes with check
gjtorikian Jan 24, 2015
fa6a998
Mention `typhoeus` and `hydra`
gjtorikian Jan 24, 2015
0c1a911
begone
gjtorikian Jan 24, 2015
a27f423
Rename favicon option
gjtorikian Jan 24, 2015
590ed99
You don't like empty values? Me either.
gjtorikian Jan 24, 2015
7006132
Add tests for Utils
Jan 24, 2015
7408a1b
Single quote style
Jan 24, 2015
41558f4
Wording
Jan 24, 2015
b04ae08
Simplify the Parallel example
Jan 24, 2015
ea18ff0
Fix Typhoeus link example
gjtorikian Jan 25, 2015
a42d479
Properly set ivar for data-proofer-ignore
gjtorikian Jan 25, 2015
b4bfe13
Proper pass of `followlocation`
gjtorikian Jan 25, 2015
6c49e6b
Rename ignore check
gjtorikian Jan 25, 2015
abb4e25
Proper check for `followlocation`
gjtorikian Jan 25, 2015
5e82ae6
I am somewhat certain this test is wrong
gjtorikian Jan 25, 2015
e7c5ca7
Emit the correct colors
gjtorikian Jan 25, 2015
778820f
Pretty sure `stdout` is rubbish here?
gjtorikian Jan 25, 2015
347d2fc
Update HTML tests
gjtorikian Jan 25, 2015
972fe5a
Update syntax
gjtorikian Jan 26, 2015
cb5ca18
Proper grammar!
gjtorikian Jan 26, 2015
146d650
Revamp and test command binary
gjtorikian Jan 26, 2015
13a3768
Try unblocking Travis?
gjtorikian Jan 26, 2015
c1fdc03
Wrap this cmd in quotes
gjtorikian Jan 27, 2015
7ff2ebf
Simplify this test, since it fails only on Travis
gjtorikian Jan 27, 2015
f8dd45d
Try to improve on sorting logic
gjtorikian Jan 27, 2015
05439f7
Use `module_function`
gjtorikian Jan 27, 2015
d93234b
Pull sorting and reporting out
gjtorikian Jan 27, 2015
2929aaf
Be green
gjtorikian Jan 27, 2015
bab216f
No nakey
gjtorikian Jan 27, 2015
731dc20
Rename `get_checks` to just `checks`
gjtorikian Jan 28, 2015
ff40de1
More aesthetics
gjtorikian Jan 28, 2015
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Gemfile
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
source "http://rubygems.org"
source 'http://rubygems.org'

gemspec
99 changes: 55 additions & 44 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,43 @@ Or install it yourself as:

**NOTE:** When installation speed matters, set `NOKOGIRI_USE_SYSTEM_LIBRARIES` to `true` in your environment. This is useful for increasing the speed of your Continuous Integration builds.

## Real-life examples

Project | Repository
:--- | :---
[Raspberry Pi documentation](http://www.raspberrypi.org/documentation/) | [raspberrypi/documentation]( https://github.com/raspberrypi/documentation)
[Open Whisper Systems website](https://whispersystems.org/) | [WhisperSystems/whispersystems.org](https://github.com/WhisperSystems/whispersystems.org)
[Jekyll website](http://jekyllrb.com/) | [jekyll/jekyll](https://github.com/jekyll/jekyll)

## What's Tested?

### Images

`img` elements:

* Whether all your images have alt tags
* Whether your internal image references are not broken
* Whether external images are showing

### Links

`a`, `link` elements:

* Whether your internal links are not broken; this includes hash references (`#linkToMe`)
* Whether external links are working

### Scripts

`script` elements:

* Whether your internal script references are not broken
* Whether external scripts are loading

### HTML

Nokogiri looks at the markup and [provides errors](http://www.nokogiri.org/tutorials/ensuring_well_formed_markup.html) when parsing your document.
This is an optional feature, set the `validate_html` option to enable validation errors from Nokogiri.

## Usage

### Using in a script
Expand Down Expand Up @@ -90,43 +127,6 @@ Don't have or want a `Rakefile`? You _could_ also do something like the followin
htmlproof ./_site
```

### Real-life examples

Project | Repository
:--- | :---
[Raspberry Pi documentation](http://www.raspberrypi.org/documentation/) | [raspberrypi/documentation]( https://github.com/raspberrypi/documentation)
[Open Whisper Systems website](https://whispersystems.org/) | [WhisperSystems/whispersystems.org](https://github.com/WhisperSystems/whispersystems.org)
[Jekyll website](http://jekyllrb.com/) | [jekyll/jekyll](https://github.com/jekyll/jekyll)

## What's Tested?

### Images

`img` elements:

* Whether all your images have alt tags
* Whether your internal image references are not broken
* Whether external images are showing

### Links

`a`, `link` elements:

* Whether your internal links are not broken; this includes hash references (`#linkToMe`)
* Whether external links are working

### Scripts

`script` elements:

* Whether your internal script references are not broken
* Whether external scripts are loading

### HTML

Nokogiri looks at the markup and [provides errors](http://www.nokogiri.org/tutorials/ensuring_well_formed_markup.html) when parsing your document.
This is an optional feature, set the `validate_html` option to enable validation errors from Nokogiri.

## Configuration

The `HTML::Proofer` constructor takes an optional hash of additional options:
Expand All @@ -147,16 +147,18 @@ The `HTML::Proofer` constructor takes an optional hash of additional options:
| `validate_html` | Enables HTML validation errors from Nokogiri | `false` |
| `check_external_hash` | Checks whether external hashes exist (even if the website exists). This slows the checker down. | `false` |

### Configuring Typhoeus
### Configuring Typhoeus and Hydra

You can also pass in any of Typhoeus' options for the external link check. For example:

``` ruby
HTML::Proofer.new("out/", {:ext => ".htm", :verbose => true, :ssl_verifyhost => 2 })
HTML::Proofer.new("out/", {:ext => ".htm", :typhoeus => { :verbose => true, :ssl_verifyhost => 2 } })
```

This sets `HTML::Proofer`'s extensions to use _.htm_, and gives Typhoeus a configuration for it to be verbose, and use specific SSL settings. Check [the Typhoeus documentation](https://github.com/typhoeus/typhoeus#other-curl-options) for more information on what options it can receive.

You can similarly pass in a `:hydra` option with a hash configuration for Hydra.

### Configuring Parallel

[Parallel](https://github.com/grosser/parallel) is being used to speed things up a bit. You can pass in any of its options with the options "namespace" `:parallel`. For example:
Expand Down Expand Up @@ -185,14 +187,23 @@ bin/htmlproof www.google.com,www.github.com --as-links

Add the `data-proofer-ignore` attribute to any tag to ignore it from the checks.


``` html
<a href="http://notareallink" data-proofer-ignore>Not checked.</a>
```

## Custom tests

Want to write your own test? Sure! Just create two classes--one that inherits from `HTML::Proofer::Checkable`, and another that inherits from `HTML::Proofer::Checks::Check`. `Checkable` defines various helper methods for your test, while `Checks::Check` actually runs across your content. `Checks::Check` should call `self.add_issue` on failures, to add them to the list.
Want to write your own test? Sure! Just create two classes--one that inherits from `HTML::Proofer::Runner`, and another that inherits from `HTML::Proofer::Checkable`.

The `Runner` subclass must define one method called `run`. This is called on your content, and is responsible for performing the validation on whatever elements you like. When you catch a broken issue, call `add_issue(message)` to explain the error.

The `Checkable` subclass defines various helper methods you can use as part of your test. Usually, you'll want to instantiate it within `run`. You have access to all of your element's attributes.

Here's an example custom test that protects against `mailto` links that point to `octocat@github.com`:

``` ruby
class OctocatLink < ::HTML::Proofer::Checkable
class OctocatLinkCheck < ::HTML::Proofer::Checkable

def mailto?
return false if @data_ignore_proofer || @href.nil? || @href.empty?
Expand All @@ -205,14 +216,14 @@ class OctocatLink < ::HTML::Proofer::Checkable

end

class MailToOctocat < ::HTML::Proofer::Checks::Check
class MailToOctocat < ::HTML::Proofer::Runner

def run
@html.css('a').each do |l|
link = OctocatLink.new l, "octocat_link", self
link = OctocatLinkCheck.new l, "octocat_link", self

if link.mailto? && link.octocat?
return self.add_issue("Don't email the Octocat directly!")
return add_issue("Don't email the Octocat directly!")
end
end
end
Expand Down
10 changes: 4 additions & 6 deletions Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,10 @@ task :proof_readme do
require 'redcarpet'

redcarpet = Redcarpet::Markdown.new Redcarpet::Render::HTML.new({}), {}
html = redcarpet.render File.open("README.md").read
html = redcarpet.render File.read('README.md')

mkdir_p "out"
File.open "out/README.html", File::CREAT|File::WRONLY do |file|
file.puts html
end
mkdir_p 'out'
File.write('out/README.html', html)

HTML::Proofer.new("./out").run
HTML::Proofer.new('./out').run
end
46 changes: 23 additions & 23 deletions bin/htmlproof
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
#!/usr/bin/env ruby
STDOUT.sync = true

$:.unshift File.join(File.dirname(__FILE__), *%w{ .. lib })
$LOAD_PATH.unshift File.join(File.dirname(__FILE__), *%w( .. lib ))

require 'html/proofer'
require 'mercenary'
require "rubygems"
require 'rubygems'

Mercenary.program(:htmlproof) do |p|
p.version Gem::Specification::load(File.join(File.dirname(__FILE__), "..", "html-proofer.gemspec")).version
p.description "Test your rendered HTML files to make sure they're accurate."
p.version Gem::Specification.load(File.join(File.dirname(__FILE__), '..', 'html-proofer.gemspec')).version
p.description %(Test your rendered HTML files to make sure they're accurate.)
p.syntax 'htmlproof PATH [options]'

p.description "Runs the HTML-Proofer suite on the files in PATH"
p.description 'Runs the HTML-Proofer suite on the files in PATH'

p.option 'ext', '--ext EXT', 'The extension of your HTML files (default: `.html`)'
p.option 'favicon', '--favicon', 'Enables the favicon checker (default: `false`).'
Expand All @@ -29,31 +29,31 @@ Mercenary.program(:htmlproof) do |p|
p.option 'check_external_hash', '--check_external_hash', 'Checks whether external hashes exist (even if the website exists). This slows the checker down (default: `false`).'

p.action do |args, opts|
args = ["."] if args.empty?
args = ['.'] if args.empty?
path = args.first

options = {}
options[:ext] = opts["ext"] unless opts["ext"].nil?
unless opts["swap"].nil?
options[:ext] = opts['ext'] unless opts['ext'].nil?
unless opts['swap'].nil?
options[:href_swap] = {}
opts["swap"].each do |s|
pair = s.split(":")
options[:href_swap][%r{#{pair[0]}}] = pair[1]
opts['swap'].each do |s|
pair = s.split(':')
options[:href_swap][/#{pair[0]}/] = pair[1]
end
end

options[:href_ignore] = opts["href_ignore"] unless opts["href_ignore"].nil?
options[:alt_ignore] = opts["alt_ignore"] unless opts["alt_ignore"].nil?
options[:file_ignore] = opts["file_ignore"] unless opts["file_ignore"].nil?
options[:disable_external] = opts["disable_external"] unless opts["disable_external"].nil?
options[:only_4xx] = opts["only_4xx"] unless opts["only_4xx"].nil?
options[:favicon] = opts["favicon"] unless opts["favicon"].nil?
options[:verbose] = opts["verbose"] unless opts["verbose"].nil?
options[:directory_index_file] = opts["directory_index_file"] unless opts["directory_index_file"].nil?
options[:validate_html] = opts["validate_html"] unless opts["validate_html"].nil?
options[:check_external_hash] = opts["check_external_hash"] unless opts["check_external_hash"].nil?

path = path.delete(' ').split(",") if opts["as-links"]
options[:href_ignore] = opts['href_ignore'] unless opts['href_ignore'].nil?
options[:alt_ignore] = opts['alt_ignore'] unless opts['alt_ignore'].nil?
options[:file_ignore] = opts['file_ignore'] unless opts['file_ignore'].nil?
options[:disable_external] = opts['disable_external'] unless opts['disable_external'].nil?
options[:only_4xx] = opts['only_4xx'] unless opts['only_4xx'].nil?
options[:favicon] = opts['favicon'] unless opts['favicon'].nil?
options[:verbose] = opts['verbose'] unless opts['verbose'].nil?
options[:directory_index_file] = opts['directory_index_file'] unless opts['directory_index_file'].nil?
options[:validate_html] = opts['validate_html'] unless opts['validate_html'].nil?
options[:check_external_hash] = opts['check_external_hash'] unless opts['check_external_hash'].nil?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this be DRY'd?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ugh probably. I was thinking about removing the bin entirely but I guess that's Not The Right Thing To Do ™️.


path = path.delete(' ').split(',') if opts['as-links']

HTML::Proofer.new(path, options).run
end
Expand Down
42 changes: 21 additions & 21 deletions html-proofer.gemspec
Original file line number Diff line number Diff line change
@@ -1,31 +1,31 @@
# -*- encoding: utf-8 -*-
$:.push File.expand_path("../lib", __FILE__)
$LOAD_PATH.push File.expand_path('../lib', __FILE__)
require 'html/proofer/version'

Gem::Specification.new do |gem|
gem.name = "html-proofer"
gem.name = 'html-proofer'
gem.version = HTML::Proofer::VERSION
gem.authors = ["Garen Torikian"]
gem.email = ["gjtorikian@gmail.com"]
gem.description = %q{Test your rendered HTML files to make sure they're accurate.}
gem.summary = %q{A set of tests to validate your HTML output. These tests check if your image references are legitimate, if they have alt tags, if your internal links are working, and so on. It's intended to be an all-in-one checker for your documentation output.}
gem.homepage = "https://github.com/gjtorikian/html-proofer"
gem.license = "MIT"
gem.executables = ["htmlproof"]
gem.authors = ['Garen Torikian']
gem.email = ['gjtorikian@gmail.com']
gem.description = %(Test your rendered HTML files to make sure they're accurate.)
gem.summary = %(A set of tests to validate your HTML output. These tests check if your image references are legitimate, if they have alt tags, if your internal links are working, and so on. It's intended to be an all-in-one checker for your documentation output.)
gem.homepage = 'https://github.com/gjtorikian/html-proofer'
gem.license = 'MIT'
gem.executables = ['htmlproof']
gem.files = `git ls-files`.split($/)
gem.test_files = gem.files.grep(%r{^(spec)/})
gem.require_paths = ["lib"]
gem.require_paths = ['lib']

gem.add_dependency "mercenary", "~> 0.3.2"
gem.add_dependency "nokogiri", "~> 1.5"
gem.add_dependency "colored", "~> 1.2"
gem.add_dependency "typhoeus", "~> 0.6.7"
gem.add_dependency "yell", "~> 2.0"
gem.add_dependency "parallel", "~> 1.3"
gem.add_dependency "addressable", "~> 2.3"
gem.add_dependency 'mercenary', '~> 0.3.2'
gem.add_dependency 'nokogiri', '~> 1.5'
gem.add_dependency 'colored', '~> 1.2'
gem.add_dependency 'typhoeus', '~> 0.6.7'
gem.add_dependency 'yell', '~> 2.0'
gem.add_dependency 'parallel', '~> 1.3'
gem.add_dependency 'addressable', '~> 2.3'

gem.add_development_dependency "redcarpet"
gem.add_development_dependency "rspec", "~> 3.1.0"
gem.add_development_dependency "rake"
gem.add_development_dependency "awesome_print"
gem.add_development_dependency 'redcarpet'
gem.add_development_dependency 'rspec', '~> 3.1'
gem.add_development_dependency 'rake'
gem.add_development_dependency 'awesome_print'
end
Loading