Unclog ttw tests #4391

chadwhitacre · 2017-03-24T18:29:29Z

chadwhitacre · 2017-03-24T18:30:51Z

❗️

$ phantomjs --version
1.9.8

chadwhitacre · 2017-03-24T18:33:09Z

travis-ci/travis-ci#3225 💃

chadwhitacre · 2017-03-24T18:35:01Z

Workarounds:

http://sepp.io/2016/06/run-phantomjs-2-1-1-on-travisci/

https://mediocre.com/forum/topics/phantomjs-2-and-travis-ci-we-beat-our-heads-against-a-wall-so-you-dont-have-to

chadwhitacre · 2017-03-24T18:41:20Z

It looks like Travis is already using 2.1.1 by default on the dist: trusty instances. I'm having the intermittent PhantomJS crashes with 2.1.1 as well.

travis-ci/travis-ci#3225 (comment) (an hour ago :o)

chadwhitacre · 2017-03-24T19:01:31Z

Could also potentially switch to Firefox (post-#4322 :o).

chadwhitacre · 2017-03-24T19:05:35Z

The failures w/ 2.0.0 I think I'm also seeing locally w/ 2.1.1. Looking into it ...

chadwhitacre · 2017-03-24T19:08:10Z

Didn't I already post this somewhere?

Traceback (most recent call last):
  File "/Users/whit537/personal/gratipay/gratipay.com/tests/ttw/test_trim.py", line 14, in test_trim_strips_all_unicode
    assert self.trim('Úaø¶') == 'a'
  File "/Users/whit537/personal/gratipay/gratipay.com/tests/ttw/test_trim.py", line 10, in trim
    return self.js("window.Gratipay.trim('{}')".format(val))
  File "/Users/whit537/personal/gratipay/gratipay.com/gratipay/testing/browser.py", line 73, in js
    return self.evaluate_script(code)
  File "/Users/whit537/personal/gratipay/gratipay.com/env/lib/python2.7/site-packages/splinter/driver/webdriver/__init__.py", line 199, in evaluate_script
    return self.driver.execute_script("return %s" % script)
  File "/Users/whit537/personal/gratipay/gratipay.com/env/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 465, in execute_script
    'args': converted_args})['value']
  File "/Users/whit537/personal/gratipay/gratipay.com/env/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
    self.error_handler.check_response(response)
  File "/Users/whit537/personal/gratipay/gratipay.com/env/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response
    raise exception_class(message, screen, stacktrace)
WebDriverException: Message: {"errorMessage":"Refused to evaluate a string as JavaScript because 'unsafe-eval' is not an allowed source of script in the following Content Security Policy directive: \"script-src 'self' assets.gratipay.com 'unsafe-inline'\".\n","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"129","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:64827","User-Agent":"Python-urllib/2.7"},"httpVersion":"1.1","method":"POST","post":"{\"sessionId\": \"f9a82430-10c4-11e7-90b0-2db0b140720e\", \"args\": [], \"script\": \"return window.Gratipay.trim('\\u02daa\\u00f8\\u00b6')\"}","url":"/execute","urlParsed":{"anchor":"","query":"","file":"execute","directory":"/","path":"/execute","relative":"/execute","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/execute","queryKey":{},"chunks":["execute"]},"urlOriginal":"/session/f9a82430-10c4-11e7-90b0-2db0b140720e/execute"}}
Screenshot: available via screen

================================= 13 failed, 7 passed in 45.89 seconds ==================================
[gratipay] $

chadwhitacre · 2017-03-24T19:10:45Z

I believe our ttw tests function by injecting scripts into the page, and that is running afoul of our CSP.

chadwhitacre · 2017-03-24T19:17:16Z

Looks like 2.1.1 chokes on the CSP as 2.0.0 does, and then also on cross-domain cookies.

[gratipay] $ grep unsafe-eval log|wc -l
       4
[gratipay] $ grep 'Can only set Cookies for the current domain' log|wc -l
       9
[gratipay] $

chadwhitacre · 2017-03-24T19:21:30Z

Best suggestion in ariya/phantomjs#11337 is to run through a proxy and strip the CSP header. ariya/phantomjs#13114 is longer ...

chadwhitacre · 2017-03-24T19:23:46Z

I'm going to explore the Firefox option.

chadwhitacre · 2017-03-24T19:24:13Z

(Best workaround to date in ariya/phantomjs#13114 is also to use a proxy.)

chadwhitacre · 2017-03-24T20:30:39Z

Having pretty good luck with Firefox, though I'm having to upgrade Selenium to duck a "profile not found" error, and the latest Selenium requires this geckodriver to be installed—I'm pretty sure you need that and Firefox itself. Then I believe I'm seeing some timing bugs that I may have been seeing occasionally under Chrome anyway.

chadwhitacre · 2017-03-24T20:44:32Z

Alright, pushed dd833f2. I'm expecting some timing failures in test_team_distribution.py.

chadwhitacre · 2017-03-24T20:45:15Z

Will we need this xvfb hack?

chadwhitacre · 2017-03-24T20:47:52Z

Getting an ancient version of Firefox (38). Addon?

chadwhitacre · 2017-03-24T21:00:35Z

That's better. 👍

$ firefox --version
Mozilla Firefox 52.0.1

chadwhitacre · 2017-03-24T21:08:32Z

http://elementalselenium.com/tips/38-headless

chadwhitacre · 2017-03-24T21:15:02Z

I'm expecting some timing failures in test_team_distribution.py.

https://travis-ci.org/gratipay/gratipay.com/builds/214804197 💃

chadwhitacre · 2017-03-24T21:25:18Z

Can we use xvfb-run instead?

Yes!

travis-ci/docs-travis-ci-com#1060

chadwhitacre · 2017-03-24T21:42:19Z

I've got a plan to implement #4314 (comment), removing all time.sleep calls from the ttw tests. That should do away once and for all with the bugs cropping up here as well as #4118 → #4129 → #4290 → #4314 → #4362.

chadwhitacre · 2017-03-25T07:39:12Z

Squashed Travis flailing into 0e360f2, had been 5417051.

chadwhitacre · 2017-03-25T07:40:15Z

With 27202ac I'm expecting Travis to be green! 💃

chadwhitacre · 2017-03-25T07:59:59Z

Ready for review and merge! Note that this includes #4390.

This should squash #4362. Also: - adapt to Firefox (`Keys.ENTER` vs `\n`) - fix js memory leak in notifications (`delete $dialog`) - only open one browser; reuse between classes - git ignore geckodriver.log

chadwhitacre · 2017-03-25T10:35:36Z

Rebased now that #4390 is in. Was b680cf4.

chadwhitacre · 2017-03-27T12:39:55Z

Ready for review @rohitpaulk @nobodxbodon @JessaWitzel @kaguillera et al.

rohitpaulk · 2017-03-28T16:54:08Z

.gitignore

@@ -20,5 +20,8 @@ docs/gratipay.rst
 _vimrc_local.vim
 .transifexrc
 npm-debug.log
+
+# ttw tests


Whoopsy doopsy

Oh it's a valid comment :D I thought it was a line used for testing :P

rohitpaulk · 2017-03-28T16:59:39Z

gratipay/testing/browser.py

+        while time.time() < end_time:
+            if not self.has_element(selector):
+                return
+        raise NeverLeft()


I think this could make it a bit difficult to debug failures - (What never left?). Maybe we could make it better by adding a message saying that The element specified by #{selector} did not disappear

It looks in 3c1e47e like passing in the selector is sufficient to get it out again in a traceback.

chadwhitacre · 2017-03-29T13:28:29Z

You good here, @rohitpaulk?

rohitpaulk · 2017-03-30T06:31:04Z

Looks good to me!

chadwhitacre · 2017-03-30T08:27:31Z

!m @rohitpaulk 💃

chadwhitacre mentioned this pull request Mar 24, 2017

Travis often hangs on ttw tests #4359

Closed

chadwhitacre force-pushed the unclog-ttw-tests branch from f7e4974 to 27202ac Compare March 25, 2017 07:38

chadwhitacre added the Review label Mar 25, 2017

chadwhitacre added 3 commits March 25, 2017 06:31

Switch to Firefox at Travis

1cd5fa0

Remove time.sleep calls from ttw tests

3abccca

This should squash #4362. Also: - adapt to Firefox (`Keys.ENTER` vs `\n`) - fix js memory leak in notifications (`delete $dialog`) - only open one browser; reuse between classes - git ignore geckodriver.log

Update README for Firefox vs. PhantomJS

6786d35

chadwhitacre force-pushed the unclog-ttw-tests branch from b680cf4 to 6786d35 Compare March 25, 2017 10:35

This was referenced Mar 25, 2017

Send confirmation email #4335

Merged

Integrate initial package claiming PRs #4305

Merged

Remove count of projects in each category #4392

Merged

rohitpaulk reviewed Mar 28, 2017

View reviewed changes

Make it a bit easier to debug

3c1e47e

chadwhitacre mentioned this pull request Mar 29, 2017

Mark dead letters dead and skip them #4395

Merged

rohitpaulk merged commit f9eb6df into master Mar 30, 2017

chadwhitacre mentioned this pull request Mar 30, 2017

Fix timing bug once and for all #4362

Closed

chadwhitacre mentioned this pull request Apr 21, 2017

Unfrobulate Travis #4413

Closed

chadwhitacre deleted the unclog-ttw-tests branch June 15, 2017 19:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unclog ttw tests #4391

Unclog ttw tests #4391

chadwhitacre commented Mar 24, 2017 •

edited

Loading

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017 •

edited

Loading

chadwhitacre commented Mar 24, 2017 •

edited

Loading

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017 •

edited

Loading

chadwhitacre commented Mar 25, 2017 •

edited

Loading

chadwhitacre commented Mar 25, 2017

chadwhitacre commented Mar 25, 2017

chadwhitacre commented Mar 25, 2017

chadwhitacre commented Mar 27, 2017

rohitpaulk Mar 28, 2017

rohitpaulk Mar 28, 2017

rohitpaulk Mar 28, 2017 •

edited

Loading

chadwhitacre Mar 29, 2017

chadwhitacre commented Mar 29, 2017

rohitpaulk commented Mar 30, 2017

chadwhitacre commented Mar 30, 2017

Unclog ttw tests #4391

Unclog ttw tests #4391

Conversation

chadwhitacre commented Mar 24, 2017 • edited Loading

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017 • edited Loading

chadwhitacre commented Mar 24, 2017 • edited Loading

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017

chadwhitacre commented Mar 24, 2017 • edited Loading

chadwhitacre commented Mar 25, 2017 • edited Loading

chadwhitacre commented Mar 25, 2017

chadwhitacre commented Mar 25, 2017

chadwhitacre commented Mar 25, 2017

chadwhitacre commented Mar 27, 2017

rohitpaulk Mar 28, 2017

Choose a reason for hiding this comment

rohitpaulk Mar 28, 2017

Choose a reason for hiding this comment

rohitpaulk Mar 28, 2017 • edited Loading

Choose a reason for hiding this comment

chadwhitacre Mar 29, 2017

Choose a reason for hiding this comment

chadwhitacre commented Mar 29, 2017

rohitpaulk commented Mar 30, 2017

chadwhitacre commented Mar 30, 2017

chadwhitacre commented Mar 24, 2017 •

edited

Loading

chadwhitacre commented Mar 24, 2017 •

edited

Loading

chadwhitacre commented Mar 24, 2017 •

edited

Loading

chadwhitacre commented Mar 24, 2017 •

edited

Loading

chadwhitacre commented Mar 25, 2017 •

edited

Loading

rohitpaulk Mar 28, 2017 •

edited

Loading