-
Notifications
You must be signed in to change notification settings - Fork 17
Theme testing
The theme uses Intern for unit testing JavaScript. We have used jstestdriver and Doh in the past so there are some unusual constructs lying around in some of the tests.
In the previous tests we had flakey UI testing. We have not yet got around to porting all those flakey tests to Intern so our actual coverage is a bitlot rubbish in the UI space. Feel free to fix this.
- Contents
- Setting up the tests
- SauceLabs
- Testing with local Selenium
- Other tunnels
- Browser runner
- Further information
The test infrastructure is built in Maven. From the wcomponents-theme directory call mvn test. This will build tests to /PROJECT_PATH/wcomponents-theme/target/test-classes/. The test run configuration is in that directory in the file intern.js.
The default set up will build the test infrastructure but not run any tests because we expect them to run in SauceLabs and don't want to share our secrets. Instead the console will provide instructions to enable you to run a local jetty instance and test in whatever browser(s) you have to hand.
The test configuration is done by setting ANT properties in your user.xml file - see Theme build options for more details of this configuration file.
The following table shows the ANT properties which may be overridden to change the way the JavaScript tests are built and run.
| Property | Default | Docs |
|---|---|---|
| intern.capabilities | recordVideo: false, recordScreenshots: false | capabilities |
| intern.tunnel | SauceLabsTunnel | tunnel |
| intern.tunnel.options | verbose: true | tunneloptions |
| intern.reporters | "" |
reporters |
| test.environments | {browserName:'firefox'},{browserName:'internet explorer',version:'11.0',platform:'Windows 10'},{browserName:'chrome'} | environments |
The default setup uses the SauceLabsTunnel but will not run tests. To have tests run on SauceLabs two environment variables must be set. Good CI Systems (such as CircleCI or Travis CI) allow these to be set so your tests run on SauceLabs using these variables.
The variables which must be set are SAUCE_USERNAME and SAUCE_ACCESS_KEY. See Intern docs for more information.
If there selenium standalone is installed on the test device the test setup can be configured to automatically run in installed browsers using the tunnel SeleniumTunnel.
The property intern.tunnel should be set to SeleniumTunnel and test.environments should be changed to suit the test device. To get coverage reports set the intern.reporters property.
NOTE To successfully run (recent) selenium-standalone requires Java8; practically, this means to run mvn test and have it run a Selenium tunnel requires a Java8 JDK. Since the purpose of this is to run JavaScript tests not Java tests this doesn't really matter.
The following shows a set of properties which will run an installed selenium server and test in Firefox with test run coverage reporting to an existing location.
<property name="intern.tunnel" value="SeleniumTunnel" />
<property name="test.environments"
value="{ browserName: 'firefox' }"/>
<property name="intern.reporters"
value="reporters: [{id: 'LcovHtml', directory: 'target/reports/'}],"/>The tests will run on the local device when the tunnel is SeleniumTunnel, as described above. The tunnel NullTunnel can be used along with a local selenium server to run the tests at-will rather than as part of the Maven test target.
This can be useful when debugging as it allows, for example, one to hack the built intern.js file. If the intern.tunnel property is NullTunnel the output of mvn test will include a line giving the command to run the tests using node. This pre-supposes that a driver is available such as a running instance of selenium-server.
The following shows a set of properties which will set up tests to be run manually and test in Firefox with test run reporting to a text file.
<property name="intern.tunnel" value="NullTunnel" />
<property name="test.environments"
value="{ browserName: 'firefox' }"/>
<property name="intern.reporters"
value="reporters: [{id: 'Runner', filename: 'target/runner.txt'}],"/>Once the configuration has been added to user.xml and mvn test has completed the node command is in the output of mvn test (towards the end) (note PATH obfuscated).
[echo] ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠
[echo] Skipping unit tests because you have set a NullTunnel:
[echo]
[echo] To manually start the tests try:
[echo] node PATH/runner.js config=target/test-classes/wcomponents-theme/intern
[echo] ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠ ☠
The following is an example bash command assuming the working directory is the wcomponents-theme base directory and no global install of node:
node/node node_modules/intern/runner.js config=target/test-classes/wcomponents-theme/intern
The tunnel NullTunnel can connect tests to a remote selenium server instance or selenium grid (or other manual connection). For more information about this and other in-built tunnels: see the Intern documentation for more information.
The Intern browser runner is useful for quick tests of tests and debugging but may give results which differ from automated or remote testing, especially if the test relies on scroll or positioned elements.
The output of mvn test always includes instructions for running the browser runner. These may be lost in the depths of the buffer unless the tunnel is NullTunnel or SauceLabsTunnel. Jetty is included to make running the tests in browser runner easier. After successfully running mvn test the next step is to start jetty from Maven and then connect to the tests - the URL is part of the output. An example follows:
> mvn test &>~/some-file.txt
> mvn jetty:start &
The URL to connect to the BrowserRunner is http://localhost:8080/target/.
NOTE that if the intern configuration has included reporters which are not compatible with the BrowserRunner your attempt to run the tests in a browser will fail. This can be overcome, in the short term, by hacking the built intern.js file.
After running the tests stop the jetty instance: mvn jetty:stop if the process was backgrounded or any of the usual methods for killing a foreground process.