-
Notifications
You must be signed in to change notification settings - Fork 143
Chutzpah.json Settings File
The Chutzpah settings file is an .json file that you can place anywhere in your testing directories to inform Chutzpah of test specific settings. The Chuzpah settings file is completely optional and if the file or any of its properties are omitted Chutzpah will use the default values.
When Chutzpah is run on a file (lets say test.js) it scans the current directory for a file named chutzpah.json. If it does not find one it will recursively traverse up the directory chain looking for one. This behavior provides a lot of flexibility since you can provide multiple chutzpah.json files in your tree and each one will work over its sub-folder. This lets you have different settings for different groups of tests.
Important: The deserializer Chutzpah uses is is restrictive about what you put in your chutzpah.json file. To make sure it parses correctly make sure you surround all properties in quotes and do not put comments in the file.
The settings file is a text file with json content of the following format:
{
"Framework": "qunit|jasmine|mocha",
"FrameworkVersion": "",
"EnableTestFileBatching": "true|false",
"InheritFromParent": "true|false",
"InheritFromPath": "<Path to another chutzpah.json>",
"IgnoreResourceLoadingErrors": "true|false"
"TestFileTimeout": "<Timeout in milliseconds>",
"TestHarnessLocationMode": "TestFileAdjacent|SettingsFileAdjacent|Custom",
"TestHarnessDirectory": "<Path to a folder>",
"TestHarnessReferenceMode": "Normal|AMD",
"RootReferencePathMode":"DriveRoot|SettingsFileDirectory",
"CodeCoverageIncludes": [],
"CodeCoverageExcludes": [],
"CodeCoverageIgnores": [],
"CodeCoverageExecutionMode": "Manual|Always|Never",
"CodeCoverageSuccessPercentage": <Number from 0 to 100>,
"CodeCoverageTimeout": <Timeout in milliseconds>
"References": [],
"Tests": [],
"CustomTestHarnessPath": "<Path to custom test harness file>",
"Compile": <A compile configuration object>,
"Server": <A server configuration object>,
"TestPattern": "<Regex test name pattern>",
"AMDBaseUrl": "<Path to same location that your Require.js config baseurl points to>",
"AMDAppDirectory": "<The root folder for your AMD paths>",
"UserAgent": "<Custom user agent to use during testing>",
"Transforms": [],
"EnableTracing": true|false,
"TraceFilePath": "<Path to log file. Defaults to %temp%/chutzpah.log>",
"Parallelism": "<Max degree of parallelism for Chutzpah. Defaults to number of CPUs>",
"BrowserArguments": <A map of browser name (keys) to corresponding browser arguments (values), i.e.; { 'chrome': '--allow-file-access-from-files' }.>
}
The chutzpah.json file supports the following properties
Framework – Determines what testing framework to use. This will override the other detection methods. This is helpful if Chutzpah is having trouble figuring out what your intended framework is. If left out Chutzpah will automatically try to detect the framework.
FrameworkVersion - Tells Chutzpah if it should use a different version of on of the test frameworks than the default one. Currently, the only framework this works for is Jasmine. As of the 3.1.0 release Chutzpah default to Jasmine 2.0 but if you want to use the 1.0 line for Jasmine still pass "1" for FrameworkVersion.
EnableTestFileBatching - Batch all test files for this chutzpah.json file in one test harness. This can be a big performance win since you are not paying the cost of multiple invocations of test harness. The trade off is you lose isolation so one test can affect another. One way to use this is to create multiple chutzpah.json files for your isolation boundaries (or logical areas) then chutzpah can create one harness per area and paralleize across your areas.
InheritFromParent - Walk up the tree from the current directory to find a parent chutzpah.json file to inherit from. Most properties are simply overridden in a child chutzpah.json file if they exist in both. Howevever, for properties like References and Transforms it will add to the existing list. This helps enable the scenario where you have common references in a parent chutzpah.json file and only have area specific references in a child one. The Tests setting does not inherit.
IgnoreResourceLoadingErrors - Suppress errors that are reported when a script request to load a url (e.g. xhr/script/image) fails
References - Using the references setting
Tests - Using the tests setting
Compile - Using the compile setting
Server - Using the server setting
TestFileTimeout - The time to wait for tests in a file to finish in milliseconds
TestHarnessLocationMode – Determines where to place the generated html test harness files. The default mode is TestFileAdjacent which means the harness is placed in the same directory as the test file. SettingsFileAdjacent means it is placed in the same directory as the chutzpah.json file. Custom mode lets you specify a custom path to the directory.
TestHarnessDirectory – When TestHarnessLocationMode is set to Custom this is either the relative or absolute path to the directory where to place the test harness.
RootReferencePathMode – This property determines what directory a rooted reference path refers to. In a test file Chutzpah parses the tags to determine your test dependencies. By default (the DriveRoot setting) if you write Chutzpah will look for a.js under C:\ (assuming you are running your test in the C drive). However, if you set this property to SettingsFileDirectory Chutpah will now look for a.js relative to the location of the chutzpah.json file. This is useful if you have a large project and want to minimize the number of long reference paths.
CodeCoverageExecutionMode - If Always, forces code coverage to run always. If Manual/null/not set, allows code coverage to run if invoked using test adapter, command line or context menu options (default). If Never, forces code coverage to never run.
CodeCoverageIncludes - The collection code coverage file patterns to include in coverage. These are in glob format. If you specify none all files are included.
CodeCoverageExcludes - The collection code coverage file patterns to exclude in coverage. These are in glob format. If you specify none no files are excluded.
CodeCoverageIgnores - This is an advanced property to use if you are having issues with CodeCoverageExcludes. When you list a file in CodeCoverageExcludes, Chutzpah will not even instrument that file using the Blanket.js engine. This can give performance improvements but in some situations can change the loading order of your files. This can cause test failures for some people. In that situation, CodeCoverageIgnores is the right option. It still lets the coverage engine instrument file but the results of coverage are ignored.
CodeCoverageSuccessPercentage - The percentage of lines should be covered to show the coverage output as success or failure. By default, this is 60.
CodeCoverageTimeout - The timeout in milliseconds for how long to wait to instrument each file for code coverage. Defaults to 5000ms.
CustomTestHarnessPath - The CustomTestHarnessPath setting allows you to override the default template Chutzpah uses for the HTML test harness. This is an advanced feature which should only be used as a last resort. Read how to use it in Customizable HTML harness generation.
AMDBaseUrl - Configures Chutzpah to work in with RequireJS when you are setting a baseurl in your Require.js config. This should point to the same location. See this example.
AMDAppDirectory - This is used in addition to the AMDBaseUrl when you have compile sources. Chutzpah uses this path to determine the right relative paths for your AMD paths. Often when compiling from a language like TypeScript or CoffeeScript you will output your entire compile tree to a new directory. In this case your baseUrl will point their BUT the AMD relative paths you want should still be relative to the source directory. See this example.
TestPattern - Using the TestPattern setting
Transforms - Using the Transforms setting
This file tells Chutzpah to use QUnit and to place the test harness next to the chutzpah.json file:
{
"Framework": "qunit",
"TestHarnessLocationMode": "SettingsFileAdjacent"
}
This file tells Chutzpah to convert TypeScript to ES5 code and to place the test harness in a custom directory:
{
"TestHarnessLocationMode": "Custom",
"TestHarnessDirectory": "../Some/Folder"
}
When run with Code Coverage this file tells Chutzpah to exclude files named jquery.js and include files that have a suffix of query.js or are named dogs.js:
{
"CodeCoverageExcludes": ["*\\jquery.js"],
"CodeCoverageIncludes": ["*query.js", "*\\dog.js"]
}