Code coverage no longer run on test:build

[scottohara]

Previously, withbabel 5 we had test:src that would run karma-babel-preprocessor to transpile (including inline source maps), and karma-coverage (using the default istanbul instrumenter) would do coverage reporting.

The generated coverage metrics were against the transpiled code; and the HTML reports would show that. We used auxiliaryCommentBefore: "istanbul ignore next" in the babel preprocessor, so that babel helper code that wasn't exercised by the test suite was ignored, allowing 100% coverage.

We also had test:build that would run karma-coverage over the final built code (compiled, minified & bundled; with external source maps). This provided confidence that the code to be deployed contained no errors and 100% coverage. The downside of this was that the HTML coverage reports from this would overwrite the ones from test:src (so the report would not be broken down by the original file names; it was all reported under the bundled filename app.js)

With babel 6, auxiliaryCommentBefore stopped working; which meant that we could no longer achieve 100% coverage using the existing approaches.

Until istanbul support for sourcemaps is ready (see references below), we're using isparta to do instrumentation; and have dropped karma-babel-preprocessor (since isparta needs to see the original uncompiled source and do it's own babel/sourcemap processing to work correctly).

What this means is we can no longer do coverage reporting in test:build.

Once istanbul sourcemap support is ready (and assuming it can properly instrument code that has already been compiled and provides sourcemaps; rather than having to do the processing itself like isparta); then the plan is to move back to using karma-babel-preprocessor and istanbul as the default instrumenter, and drop isparta.

gotwarlost/istanbul#212
gotwarlost/istanbul#413
gotwarlost/istanbul#512
gotwarlost/istanbul#516

[scottohara]

This will become redundant with the move to webpack. Going foward we will only have:

• npm run test:bdd (bundles with weback and inline-source-map, and runs the full test suite in watch mode in normal Chrome with no coverage reporting). Used in development, and for debugging tests.

• npm run test (as above, but only does a single run in Headless Chrome with coverage reporting). Used in CI/CD.