Replace instanbul with nyc

[vweevers]

When we have a solid setup here, we can copy it to levelup (Level/levelup#459) and others.

[vweevers]

Some research. Goal: combine lcov reports of node and browser tests and send it to coveralls.

• nyc airtap does not work because airtap serves browserify bundles unbeknownst to nyc
• airtap collects code coverage but doesn't save it to disk (see Export Istanbul coverage defunctzombie/zuul#174, Output TAP and LCOV files for CI reporting integration defunctzombie/zuul#230 and Allow pipelining coverage data from browsers defunctzombie/zuul#217).
• It's instanbul-middleware does expose http://localhost:<port>/airtap/coverage/download which serves zipped lcov files.
• We might be able to hook into that middleware to get the lcov data from node.
• If not, we can either make a HTTP request to the above endpoint, or have airtap send window.__coverage__ to the server, which'd then save it to .nyc_output/out.json. See also browserify-instanbul, the browserify transform used by airtap.
• If multiple suites (e.g. node and browser tests) are run on the same machine, nyc automatically combines the reports. See document and test nyc merge functionality istanbuljs/nyc#515. If we can't use the nyc CLI, there are alternative tools for merging reports:
  • istanbul report json (see Deprecate in favor of istanbul report jamestalmage/istanbul-combine#2)
  • istanbul-merge
  • lcov-result-merger
  • istanbul-combine
• But we have multiple machines. We can configure Coveralls to merge results of parallel builds.
• We'll still have to combine results of parallel browsers ourselves (airtap could do this). Or write to separate lcov files in a way that's compatible with nyc.

[vweevers]

Plan:

• Remove TypeScript tests from memdown (to clean up npm scripts)
• Remove existing coverage and report-coverage npm scripts and istanbul dependencies
• Add coverage script: nyc report --reporter=text-lcov | coveralls
• Add hook to .travis.yml: after_success: npm run coverage. Each Travis node job will send lcov reports to coveralls.
• Add a feature to airtap to write LCOV files instanbul files to disk. Each browser result should be written to its own file.
• Change our Travis browser job to run npm run test-browsers && npm run report-coverage.
• Configure Coveralls to merge results of parallel builds.

[ralphtheninja]

Good work @vweevers

[vweevers]

Before attacking the airtap part, which requires writing code, we can setup code coverage for node only and test parallel builds.

[vweevers]

Nice, nyc doesn't seem to care about the .nyc-output/*.json filenames. We could write .nyc-output/airtap-001.json, .nyc-output/airtap-002.json or perhaps .nyc-output/<browser>-<version>-<os>.json, and nyc report still works.

[vweevers]

Waiting on airtap/airtap#194