Provide complete example of using caching on CI

[fulldecent]

Please provide a complete example of using the external file fetching feature in conjunction with Travis results caching.

(I will provide resources to work on this issue. Just wanted to add the issue here to see if this project will be interested to accept such a change when it is available.)

[fulldecent]

Here is documentation on Travis caching: https://docs.travis-ci.com/user/caching/

To cache arbitrary directories, see https://docs.travis-ci.com/user/caching/#Arbitrary-directories

cache:
  directories:
  - .autoconf
  - $HOME/.m2

[fulldecent]

Here is documentation on html-proofer caching:

https://github.com/gjtorikian/html-proofer/blob/master/bin/htmlproofer#L40

Just use --timeframe 6w or similar.

Then the results pulled from / saved to ./tmp/.htmlproofer

[fulldecent]

FYI, we are trying this and it does not work:

language: ruby
rvm:
- 2.2.5

script:
 - bundle exec jekyll build
 - bundle exec htmlproofer --check-external-hash --check-favicon --check-html --check-img-http --check-opengraph --enforce-https --timeframe 6w ./_site

cache:
  directories:
  - $HOME/tmp/.htmlproofer #https://github.com/gjtorikian/html-proofer/issues/381
  
env:
  global:
  - NOKOGIRI_USE_SYSTEM_LIBRARIES=true # speeds up installation of html-proofer

sudo: false # route your build to the container-based infrastructure for a faster build

See (private URLs)

https://travis-ci.com/fulldecent/mtssites/builds/40171680
https://travis-ci.com/fulldecent/mtssites/builds/40171269

@ilyalyo

[VirenMohindra]

Adding onto this @gjtorikian, another way of maybe making this work is explicitly stating the absolute path as a flag

eg:

 - bundle exec htmlproofer --timeframe 6w  --cache-path "/tmp/.htmlproofer/cache.log" ./_site

My travis build is failing because I'm getting a timeout error.

Or maybe not cache in a hidden folder?

Another option would be to retry the link several times (another flag) --retry-link-check 3

[fulldecent]

Here are some relevant parts from the travis build I can share:

Cache is fetched successfully

$ export CASHER_DIR=$HOME/.casher
$ Installing caching utilities

attempting to download cache archive
fetching acls.net/cache-linux-precise-XXXXXXXX--rvm-2.2.5--gemfile-Gemfile.tgz
found cache

adding /home/travis/tmp/.htmlproofer to cache
creating directory /home/travis/tmp/.htmlproofer

htmlproofer runs

$ bundle exec htmlproofer --check-external-hash --check-favicon --check-html --check-img-http --check-opengraph --enforce-https --timeframe 6w ./_site
Running ["LinkCheck", "HtmlCheck", "FaviconCheck", "ImageCheck", "ScriptCheck", "OpenGraphCheck"] on ["./_site"] on *.html... 
Found 0 links in the cache...
Adding 170 links to the cache...
Removing 0 links from the cache...
Checking 170 external links...
Ran on 32 files!

Cache is stored successfully

store build cache
0.00s
1.72schanges detected, packing new archive
uploading archive

---

I run this a few times and it is always Found 0 links in the cache...

[ilyalyo]

Added fix in this commit: https://github.com/fulldecent/mtssites/commit/b26339162bc8c6ef5a9809c7e42b3c86a07fd69c
The problem was not connected with htmlprofer, it was wrong envirement variable - $HOME is travis user home dir, TRAVIS_BUILD_DIR is a project dir.

Found 170 links in the cache...
Adding 1 link to the cache...
Removing 1 link from the cache...
Checking 28 external links...
Ran on 32 files!

Private build:
https://travis-ci.com/fulldecent/mtssites/builds/40646884#L239-L243

[VirenMohindra]

@ilyalyo it worked! thank you so much. Issue can be closed

[fulldecent]

Thanks @ilyalyo, that fixes the problem for MTSSITES.

As for helping this project, now we have a kick-ass implementation that we can recommend:

language: ruby
rvm:
- 2.2.5

script:
 - bundle exec jekyll build
 - bundle exec htmlproofer --check-external-hash --check-favicon --check-html --check-img-http --check-opengraph --enforce-https --timeframe 6w ./_site

cache:
  directories:
  - $TRAVIS_BUILD_DIR/tmp/.htmlproofer #https://github.com/gjtorikian/html-proofer/issues/381
  
env:
  global:
  - NOKOGIRI_USE_SYSTEM_LIBRARIES=true # speeds up installation of html-proofer

sudo: false # route your build to the container-based infrastructure for a faster build

[fulldecent]

Please keep the issue open. I am working on finding somewhere to publish this learning.

[gjtorikian]

❤️ Thanks y'all. I'll add this information to the README.

[gjtorikian]

As the original issue is resolved, and the relevant lines are in the README, I am going to close the issue now though, to keep the open bug list relevant. ✌️

It should be easy to bookmark / find one more if you need it again!