Code Coverage in Chutzpah
Chutzpah’s new code coverage features is built on top of the great open source library Blanket.js. When you run with code coverage enabled it will use Blanket.js to instrument the files under test and gather the coverage results. You can perform code coverage from the command line and both Visual Studio plugins.
Chutzpah uses Blanket.js to do code coverage. However, Blanket.js does not support System.js. So, generating coverage for Angular2 will not work :(
To get code coverage from the command line you pass the /coverage flag to chutzpah.console.exe.
chutzpah.console.exe /coverage /path test.js
After this command finishes you will have a two files placed in your current directory
_Chutzpah.coverage.html – The rendered html results
_Chutzpah.coverage.json – The raw json results
When you open the results html file you will see something like:
The Visual Studio Context Menu Extension now contains a Show Code Coverage command which will perform code coverage and open the results in a browser.
For the Unit Test Explorer adapter for Visual Studio 2012 Chutzpah piggybacks on the existing “Analyze Code Coverage” menu option and uses it to run with code coverage and show the results in a browser. Ideally, Chutzpah would integrate into the code coverage reporting built into VS but I was unable to figure out a good way to plug in to that. The side effect is that when you run this command VS will open their results window first (with nothing in it) and then Chutzpah will launch its results in a browser.
NOTE: I have noticed some performance issues when running “Analyze Code Coverage for All Tests” command. I think it might be VS doing extra work for no reason since it is trying to generate its own report.
Being able to run code coverage is not enough since in many cases the results will not be accurate. For example, if your project uses jQuery you do not expect your test to cover all the lines of jQuery’s source code (that would be one impressive test). Therefore you need to exclude files/directories from participating in code coverage. There are two way you can do that with Chutpah: command line arguments or a chutzpah.json file.
If you are running Chutzpah from the command line there are two additional arguments you can pass coverageIncludes and coverageExcludes. These are a comma separated list of files/paths you want to include or exclude from coverage. They can use a glob format where * matches zero or more characters and ? matches one character. For example:
Exclude files that ends in matt.js
chutzpah.console.exe /path test.js /coverage /coverageExcludes *matt.js
Include only dog.js and any file with a single character extension
chutzpah.console.exe /path test.js /coverage /coverageIncludes "*\dog.js, *.?"
Include *query.js but exclude jQuery.js
chutzpah.console.exe /path test.js /coverage /coverageIncludes *query.js /coverageExcludes *\jquery.js
The other option for configuring code coverage is the chutzpah.json file. There are two new setting which you can use CodeCoverageExcludes and CodeCoverageIncludes. These settings work in the same way as the their command line counterpart. The main difference is that you must escape backslashes.
{
"CodeCoverageExcludes": ["*\\jquery.js"],
"CodeCoverageIncludes": ["*query.js", "*\\dog.js"]
}
You force coverage to always be gathered by setting CodeCoverageExecutionMode to always. This is very useful when running Chutzpah in a build server.
{
"CodeCoverageExecutionMode": "Always"
}
You can force coverage to never be gathered by setting CodeCoverageExecutionMode to Never.
{
"CodeCoverageExecutionMode": "Never"
}