Spurious "Couldn't resolve host name" errors

[doismellburning]

Running html-proofer over my Octopress site, I get a bunch of "Couldn't resolve host name" errors on URLs I can "hand-resolve"

My Rakefile:

$ grep -A 10 "Run tests" Rakefile 
desc "Run tests"
task :test => [:generate] do
  HTML::Proofer.new("./public", {
    :empty_alt_ignore => true,
  }).run
end

The results:

$ bundle exec rake test
# Snip
  *  External link https://github.com/boto/boto failed: 0 Couldn't resolve host name
# Snip
rake aborted!
HTML-Proofer found 213 failures!

Tasks: TOP => test
(See full trace by running task with --trace)

But that URL poses no problems elsewhere:

$ curl -s -D - -o /dev/null https://github.com/boto/boto | head -n 1
HTTP/1.1 200 OK

My Gemfile.lock:

$ cat Gemfile.lock 
GEM
  remote: http://rubygems.org/
  specs:
    RedCloth (4.2.9)
    addressable (2.3.8)
    chunky_png (1.2.5)
    classifier (1.3.3)
      fast-stemmer (>= 1.0.0)
    colored (1.2)
    compass (0.12.2)
      chunky_png (~> 1.2)
      fssm (>= 0.2.7)
      sass (~> 3.1)
    directory_watcher (1.4.1)
    ethon (0.8.0)
      ffi (>= 1.3.0)
    fast-stemmer (1.0.1)
    ffi (1.9.10)
    fssm (0.2.9)
    haml (3.1.7)
    html-proofer (2.5.1)
      addressable (~> 2.3)
      colored (~> 1.2)
      mercenary (~> 0.3.2)
      nokogiri (~> 1.5)
      parallel (~> 1.3)
      typhoeus (~> 0.7)
      yell (~> 2.0)
    jekyll (0.12.0)
      classifier (~> 1.3)
      directory_watcher (~> 1.1)
      kramdown (~> 0.13.4)
      liquid (~> 2.3)
      maruku (~> 0.5)
      pygments.rb (~> 0.3.2)
    kramdown (0.13.8)
    liquid (2.3.0)
    maruku (0.6.1)
      syntax (>= 1.0.0)
    mercenary (0.3.5)
    mini_portile (0.6.2)
    nokogiri (1.6.6.2)
      mini_portile (~> 0.6.0)
    parallel (1.6.1)
    posix-spawn (0.3.6)
    pygments.rb (0.3.4)
      posix-spawn (~> 0.3.6)
      yajl-ruby (~> 1.1.0)
    rack (1.5.2)
    rack-protection (1.5.0)
      rack
    rake (0.9.2.2)
    rb-fsevent (0.9.1)
    rdiscount (2.0.7.2)
    rubypants (0.2.0)
    sass (3.1.20)
    sass-globbing (1.0.0)
      sass (>= 3.1)
    sinatra (1.4.2)
      rack (~> 1.5, >= 1.5.2)
      rack-protection (~> 1.4)
      tilt (~> 1.3, >= 1.3.4)
    stringex (1.4.0)
    syntax (1.0.0)
    tilt (1.3.7)
    typhoeus (0.8.0)
      ethon (>= 0.8.0)
    yajl-ruby (1.1.0)
    yell (2.0.5)

PLATFORMS
  ruby

DEPENDENCIES
  RedCloth (~> 4.2.9)
  compass (~> 0.12.2)
  haml (~> 3.1.7)
  html-proofer
  jekyll (~> 0.12)
  liquid (~> 2.3.0)
  pygments.rb (~> 0.3.4)
  rake (~> 0.9)
  rb-fsevent (~> 0.9)
  rdiscount (~> 2.0.7)
  rubypants (~> 0.2.0)
  sass-globbing (~> 1.0.0)
  sinatra (~> 1.4.2)
  stringex (~> 1.4.0)

BUNDLED WITH
   1.10.3

What am I missing...?

(Tried with Ruby 1.9.3 and 2.1.1)

[gjtorikian]

Hm. Can you link me to the site so that I can clone it and check?

[doismellburning]

http://blog.doismellburning.co.uk/

So interestingly, if I check everything I get the "Couldn't resolve" fails:

$ bundle exec rake test
## Generating Site with Jekyll
identical source/stylesheets/screen.css 
Configuration from /Users/kristian/git/blog.doismellburning.co.uk/_config.yml
Building site: source -> public
Successfully generated site: source -> public
Running ["ImageCheck", "ScriptCheck", "LinkCheck"] checks on ./public on *.html... 


Checking 413 external links...
Ran on 60 files!

# Snip

- ./public/spaceteam/index.html
  *  External link http://www.sleepingbeastgames.com/spaceteam/ failed: 0 Couldn't resolve host name

# Snip

BUT if I just check a single file:

$ mkdir -p badger/spaceteam
$ cp public/spaceteam/index.html badger/spaceteam/
$ tree badger/
badger/
└── spaceteam
    └── index.html

1 directory, 1 file
$ diff badger/spaceteam/index.html public/spaceteam/index.html
$ bundle exec htmlproof badger/
Running ["ImageCheck", "LinkCheck", "ScriptCheck"] checks on badger/ on *.html... 


Checking 11 external links...
Ran on 1 files!


- badger/spaceteam/index.html
  *  image /images/kglass_400x400.jpg does not have an alt attribute (line 202)
  *  internal image /images/kglass_400x400.jpg does not exist (line 202)
  *  internal script /javascripts/modernizr-2.0.js does not exist (line 27)
  *  internal script /javascripts/octopress.js does not exist (line 30)
  *  internally linking to /, which does not exist (line 86)
  *  internally linking to /, which does not exist (line 66)
  *  internally linking to /atom.xml, which does not exist (line 74)
  *  internally linking to /atom.xml, which does not exist (line 26)
  *  internally linking to /blog/archives, which does not exist (line 87)
  *  internally linking to /document-all-the-things/, which does not exist (line 229)
  *  internally linking to /favicon.png, which does not exist (line 24)
  *  internally linking to /permalinks/, which does not exist (line 190)
  *  internally linking to /questions-are-documentation-bugs/, which does not exist (line 187)
  *  internally linking to /removing-myself-from-the-equation/, which does not exist (line 217)
  *  internally linking to /stylesheets/screen.css, which does not exist (line 25)
  *  internally linking to /the-government-service-design-manual/, which does not exist (line 213)
  *  internally linking to /treat-your-servers-like-cattle-not-like-pets/, which does not exist (line 225)
  *  internally linking to /user-stories/, which does not exist (line 221)
htmlproof 2.5.1 | Error:  HTML-Proofer found 18 failures!

It's also worth noting that my (private) Circle tests don't exhibit this problem either.

I don't seem to have any other local resolver issues though...

[doismellburning]

Hm. I wonder if the "Couldn't resolve host name" is a total red herring, for example in the same bulk run:

- public/it-hurts-with-every-heartbleed/index.html
  *  External link http://git.openssl.org/gitweb/?p=openssl.git%3Ba=commitdiff%3Bh=4817504d069b4c5082161b02a22116ad75f822b1%3Bhp=84b6e277d4f45487377d0159e82c356d750e1218 failed: 404 No error
  *  External link http://perfectforwardsecrecy.com/ failed: 301 Couldn't resolve host name

I'm not totally convinced by the accuracy of these error messages at this point!

[gjtorikian]

Well, http://git.openssl.org is a 404, and http://perfectforwardsecrecy.com/ does send a 301 over to https, so those seem accurate.

[doismellburning]

@gjtorikian Sure, the status codes are accurate, but the messages? 404 No error and 301 Couldn't resolve host name sure look like something's wonky...!

[doismellburning]

Ah well spotted, that certainly looks very plausible, thanks! Will give it a try later, cheers :)