Browser testing #451

subtleGradient · 2013-10-29T16:09:58Z

No description provided.

benjamn · 2013-10-29T16:11:20Z

grunt/tasks/populist.js


 module.exports = function() {
  var config = this.data;
  var done = this.async();

+  var theFilesToTestScript = fs.createWriteStream(__dirname + '/../../test/the files to test.generated.js');


How confident are we about spaces in file names? I'd be tempted to just use underscores here.

subtleGradient · 2013-10-29T16:13:03Z

It's not passing every browser in testling-ci yet. I'm investigating.
Cf. https://ci.testling.com/subtleGradient/react

benjamn · 2013-10-29T16:14:27Z

src/test/worker.js

@@ -6,7 +6,7 @@ var global = {};
 importScripts("phantomjs-shims.js");

 try {
-  importScripts("react.js");
+  importScripts("../../build/react.js");


I think this requires us to run grunt build before grunt test, so that build/react.js will be built. Wondering if we should make grunt test depend on grunt build.

Yeah, I'm not 100% sure about this change. I did it so that I could get the tests running in a browser normally without having to rely on the phantomjs web server contortions.

benjamn · 2013-10-29T16:17:35Z

Can we not expect Testling-CI to run grunt for us?

benjamn · 2013-10-29T16:18:28Z

src/environment/__tests__/ReactWebWorker-test.js

    var done = false;
    var error;

-    var worker = new Worker('/worker.js');
+    var worker = new Worker(window.ReactWebWorker_URL || '/src/test/worker.js?_=' + Date.now().toString(36));


cc @spicyj who last worked with ReactWebWorker-test: thoughts?

No strong feelings, I was expecting someone to want to change the generic path earlier than now. Can we not just use a relative path here though? I think I'm actually just missing context on the purpose of this PR.

subtleGradient · 2013-10-29T16:19:37Z

I don't know how to get testling-ci to run grunt.
I do know that if you include a path in the "tests" section of the "testling" package.json thing it'll get run through browserify first. So maybe there's some way to piggyback on that somehow.

benjamn · 2013-10-29T16:21:09Z

test/browser-runner.js

+    return a.pathname;
+  }());
+
+  var __dirname = __filename.split('/').reverse().slice(1).reverse().join('/');


Since you've got your own scope here, why not just

var parts = __filename.split('/'); parts.pop(); var __dirname = parts.join('/');

benjamn · 2013-10-29T16:27:02Z

@subtleGradient can you look into why Travis CI tests are failing?

Looks like something to do with ReactWebWorker timing out: https://travis-ci.org/facebook/react/builds/13216897

subtleGradient · 2013-10-29T16:31:45Z

@benjamn looking into it now…

benjamn · 2013-10-29T16:31:49Z

test/browser-runner.js

+  document.write('<script src="' + __dirname + '/../test/the-files-to-test.generated.js' + cacheBust + '"><\/script>');
+  document.write('<script src="' + __dirname + '/../test/jasmine-execute.js' + cacheBust + '"><\/script>');
+
+}());


Not sure I see the point in making this a dynamic JS file instead of generating a simple HTML file. Am I missing something?

testling-ci can load scripts like this, but it won't load an html file.

subtleGradient · 2013-10-29T17:56:52Z

Got testling-ci working with IE10 and probably more.
React apparently fails a few tests.

subtleGradient · 2013-10-29T21:42:41Z

Switching to SauceLabs…

benjamn · 2013-10-29T21:44:29Z

👍 looks legit

subtleGradient · 2013-10-29T22:43:32Z

The grunt-saucelabs implementation of jasmineRunner seems to be broken.
I'll have to fix it.

benjamn · 2013-10-31T17:10:57Z

test/browser-runner.js

@@ -0,0 +1,37 @@
+;(function(){


Can we get rid of this file if we're using SauceLabs instead of Testling?

Still want to get rid of this file if possible.

subtleGradient · 2013-11-06T20:27:16Z

I decided to remove testling and sauce labs as dependencies for this PR. I'll add sauce labs support in a separate PR.

Not necessary now that we aren't supporting testling.

benjamn · 2013-11-06T20:40:32Z

Gruntfile.js

-  grunt.loadNpmTasks('grunt-compare-size');
-  grunt.loadNpmTasks('grunt-contrib-compress');
+  for (var key in grunt.file.readJSON("package.json").devDependencies) {
+      if (key !== "grunt" && key.indexOf("grunt") === 0) grunt.loadNpmTasks(key);


Maybe just if (key.indexOf("grunt-") === 0) grunt.loadNpmTasks(key);?

Yeah, I just pasted this from somewhere. Dang my slackerly ways!

Running the tests shouldn't make git status dirty.

benjamn · 2013-11-06T22:39:15Z

Looking great! My final request is to make grunt test --debug (same command as before) keep the server running so you can run tests in local browser.

subtleGradient · 2013-11-07T16:01:01Z

You can run tests in a local browser by opening the test/index.html file, no server necessary.
If you need to keep the localhost:9999 server running for some reason you can use grunt connect:server:keepalive.

I'm not sure how to make it so that grunt test --debug runs something similar to grunt connect:server:keepalive, but I'm sure it's possible.

subtleGradient · 2013-11-07T16:16:32Z

This is what I've been doing to test changes really quickly…

Run chromedriver in one terminal window.[1]
Turn on the localhost:9999 server using grunt connect:server:keepalive in another terminal window.
Run grunt webdriver-jasmine:local in yet another terminal window.[2]

If you also want to test changes to any of the generated files you'll need to rebuild them as necessary with grunt build or whatever more specific task. This part could be made a little faster.

[1] Any webdriver server running on localhost:9515 will be used by the webdriver-jasmine:local task. You can use chromedriver or phantomjs --webdriver=9515 or whatever else you like. The webdriver-phantomjs task is essentially the same as phantomjs --webdriver=9515 except it kills it once its done.

[2] To keep the browser window open, pass the --webdriver-keep-open flag to grunt. e.g., grunt webdriver-jasmine:local --webdriver-keep-open. You can then refresh this page occasionally to manually test changes after rebuilding.

…sting Browser testing.

benjamn reviewed Oct 29, 2013
View reviewed changes

benjamn reviewed Oct 31, 2013
View reviewed changes

subtleGradient added 11 commits November 6, 2013 15:30

Generate the browser test list at built time

001bda2

Browser test runner

c4ba8f8

serve worker.js from its actual relative path

8205c68

Format test results as TAP

b867aa0

enable testling-ci browser testing

ecd847c

skip Worker test unless the browser supports them

5640d64

include the built files for testling

1393e55

Testing-ci can't handle spaces in filenames?

24ec78f

Make it easier to repro tests in the wild

a447f53

Fixed paths for phantomjs

46c0aee

Absolute urls for less flakiness

fc57283

subtleGradient added 12 commits November 6, 2013 15:30

log using grunt

b725097

remove saucelabs for now

2b273d8

cleanup

2d979a9

grunt task launch phantomjs as a webdriver server

71772e7

wait for all results to finish sending

c780985

cleanup webdriver jasmine task

1368b29

cleanup

159d64d

keep track of incomplete

dfb4dde

add logging to webdriver config

3e2d3e4

start a local webdriver server before using it

be7ee1e

removed compiled build files

f4d487f

Not necessary now that we aren't supporting testling.

Not necessary anymore

2d6a839

benjamn reviewed Nov 6, 2013
View reviewed changes

subtleGradient added 6 commits November 6, 2013 16:43

move browser test libs to lib folder

b922d8d

replace phantom-harness runner with webdriver

c9401be

making phantomjs version less specific

19a5505

cleanup loadNpmTasks

0a120bb

ignore generated file

5fa7075

Running the tests shouldn't make git status dirty.

rename sauce-harness to index

f289e98

allow directory browsing

241d57a

benjamn added a commit that referenced this pull request Nov 7, 2013

Merge pull request #451 from subtleGradient/subtlegradient/browser-te…

b3c87ea

…sting Browser testing.

benjamn merged commit b3c87ea into facebook:master Nov 7, 2013

subtleGradient deleted the subtlegradient/browser-testing branch November 7, 2013 18:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Browser testing #451

Browser testing #451

subtleGradient commented Oct 29, 2013

benjamn Oct 29, 2013

subtleGradient commented Oct 29, 2013

benjamn Oct 29, 2013

subtleGradient Oct 29, 2013

benjamn commented Oct 29, 2013

benjamn Oct 29, 2013

sophiebits Oct 29, 2013

subtleGradient commented Oct 29, 2013

benjamn Oct 29, 2013

benjamn commented Oct 29, 2013

subtleGradient commented Oct 29, 2013

benjamn Oct 29, 2013

subtleGradient Oct 29, 2013

subtleGradient commented Oct 29, 2013

subtleGradient commented Oct 29, 2013

benjamn commented Oct 29, 2013

subtleGradient commented Oct 29, 2013

benjamn Oct 31, 2013

benjamn Nov 6, 2013

subtleGradient commented Nov 6, 2013

benjamn Nov 6, 2013

subtleGradient Nov 6, 2013

benjamn commented Nov 6, 2013

subtleGradient commented Nov 7, 2013

subtleGradient commented Nov 7, 2013

Browser testing #451

Browser testing #451

Conversation

subtleGradient commented Oct 29, 2013

Choose a reason for hiding this comment

subtleGradient commented Oct 29, 2013

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benjamn commented Oct 29, 2013

Choose a reason for hiding this comment

Choose a reason for hiding this comment

subtleGradient commented Oct 29, 2013

Choose a reason for hiding this comment

benjamn commented Oct 29, 2013

subtleGradient commented Oct 29, 2013

Choose a reason for hiding this comment

Choose a reason for hiding this comment

subtleGradient commented Oct 29, 2013

subtleGradient commented Oct 29, 2013

benjamn commented Oct 29, 2013

subtleGradient commented Oct 29, 2013

Choose a reason for hiding this comment

Choose a reason for hiding this comment

subtleGradient commented Nov 6, 2013

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benjamn commented Nov 6, 2013

subtleGradient commented Nov 7, 2013

subtleGradient commented Nov 7, 2013