Unit test implementation

[occ]

Implement a test mechanism to run unit tests on different browser/OS combinations, preferably integrated with travis/grunt.

[devinrhode2]

Checkout BusterJS, Casper, and we'll probably use the BrowserStack api.

I feel like we could do pretty lightweight unit testing with the
BrowserStack api, perhaps using something like
http://ryanseddon.github.com/bunyip/

-Devin http://zerply.com/DevinRhode2
http://zerply.com/devinrhode2

On Fri, Dec 14, 2012 at 8:09 PM, Onur Can Cakmak
notifications@gh.neting.ccwrote:

Implement a test mechanism to run unit tests on different browser/OS
combinations, preferably integrated with travis/grunt.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/11.

[devinrhode2]

So testling offers free cross-browser testing for open source projects! http://ci.testling.com

@airportyh, given that you setup the mocha tests, would you be interested in leading the way on this? It actually looks really easy.

testling also offers a TravisCL style image, as you can see in this example repository: https://github.com/substack/mocha-testling-ci-example

[airportyh]

Ooooh, testem anyone? ;) We are just in the middle of dogfooding our BrowserStack integration.

[airportyh]

Although, it is hard to argue with free :) I will take a look at using testling also and I won't play favorites.

[devinrhode2]

Yeah if you want to use your testem tool, have at it!

Being able to test in mobile browsers would via BrowserStack be great, but allowing anyone to independently cross-browser test a patch via testling is also really great - perhaps testem could make it configurable to use either one?

[airportyh]

Wrote the first unit test https://github.com/airportyh/TraceKit/blob/tests/tests/tests.js

Not yet integrated with the cloud yet, but I wanted to show you the test I wrote and get some feedback as to whether the test is readable/understandable and also get you guys thinking about what are all the cases you'd like to test. This test is passing on Chrome, Firefox, Safari, Opera and IE9 for me.

Also, on the choice of Jasmine over Mocha: I've used both previously, but it was mostly because the following thing about Mocha that I went with Jasmine: with Mocha, if you have an assertion error within an async callback, you get a timeout error not assertion error, unlike Jasmine.

[devinrhode2]

Something I've been thinking about a little is how to verify no errors are
hitting window.onerror, but I have to do some fiddling

-Devin http://zerply.com/DevinRhode2
http://zerply.com/devinrhode2

On Mon, Jan 7, 2013 at 3:03 PM, Toby Ho notifications@github.com wrote:

Wrote the first unit test
https://github.com/airportyh/TraceKit/blob/tests/tests/tests.js

Not yet integrated with the cloud yet, but I wanted to show you the test I
wrote and get some feedback as to whether the test is
readable/understandable and also get you guys thinking about what are all
the cases you'd like to test.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/11#issuecomment-11971537.

[airportyh]

Alright, so I had a go with testling and had high hopes for it, because it is really cool how all it requires is for your test framework to output the tap protocol via console.log, so I used Jasmine with the tap reporter. However, ultimately it didn't work for TraceKit because it bundles all of the source code into a string to be executed via eval, which kills stacktraces for TraceKit. I tried instead to hotlink to github for all the source files, but only to realize we are now block by the single-origin policy.

[devinrhode2]

The simplest level of testing I can think of:
0: run the file with no errors
1:
Try throwing error and call report with no errors:

try {
  try {
    throw new Error('asdf');
  } catch (e) {
    TraceKit.report(e);
  }
  //test successful
} catch (e) {
  //test failed
}

2: Do number 1 and then also make sure the subscriber was triggered (who knows with what)

There's probably other unit tests we could write, but I'm not totally sure what these should/would be, I'm not very experienced with testing in general.

Having just the above things automated would be fantastic I think.

[airportyh]

Okay, we can do that much with Testling, and I'll set that up. But I don't think that's going far enough. A proper test suite should test all of the relevant functionality. And given that TraceKit is expected to have different behavior in different browsers, that should be taken into account in the test suite as well. Being that TraceKit wasn't developed with tests up front makes that harder, but that's okay, we can work from the outside in, I've done that with projects before.

[devinrhode2]

BUT WAIT Sauce Labs is free for open source too and they support mobile browsers: http://saucelabs.com/opensource (no idea about the eval situation though)

[airportyh]

Ooo, sweet! Was just working with saucelabs last night too. Sauce will be fine, it doesn't do anything funny like eval.

[devinrhode2]

Information ala mode

TestSwarm from jQuery: https://github.com/jquery/testswarm/wiki

Some issues Tom Dale ran into a few days ago with Mocha: exporting globals (caused a false positive in one test), mocha uses exception catching for asyncs tests, so exceptions in tests get swallowed instead of bubbled. And it makes the browser behave like node, breaking env detection code. They had a hell of a time getting a project to run under node and the browser (but I don't think TraceKit needs to worry about node, the wise array of browsers are enough)

Substack replies: Check out https://github.com/substack/tape for commonj-esque browser+node tests.

Someone else mentioned moutjs: https://github.com/mout/mout/tree/master/tests

[devinrhode2]

Regarding testing stuff, I've had a terrible time with testling, but see SauceLabs is fully open for anyone to integrate an open source project: https://saucelabs.com/opensource

Will update when I put it to the test.

[niemyjski]

@airportyh, were you able to get any further on this? I'd like to get some tests implemented.

[airportyh]

@niemyjski no, I haven't done anything more since.

[niemyjski]

I kind of want to drop the current tests completely and have something like: https://github.com/exceptionless/Exceptionless.JavaScript/blob/master/src/ExceptionlessClient-spec.ts

Would be much easier to maintain than have a bunch of html files.