Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flaky test CreateAdminUsersTest#test_visiting_the_index #1538

Closed
erbridge opened this issue Nov 9, 2022 · 4 comments
Closed

Fix flaky test CreateAdminUsersTest#test_visiting_the_index #1538

erbridge opened this issue Nov 9, 2022 · 4 comments
Assignees

Comments

@erbridge
Copy link
Contributor

erbridge commented Nov 9, 2022

Description

Fix the flaky test that sometimes causes CI to fail:

Error:
CreateAdminUsersTest#test_visiting_the_index:
Capybara::ElementNotFound: Unable to find field "First name" that is not disabled
    test/system/create_admin_users_test.rb:35:in `block in <class:CreateAdminUsersTest>'
@ivan-kocienski-gfsc
Copy link
Contributor

publiclab/plots2#5683 seems to elucidate the problem and a potential solution

@erbridge
Copy link
Contributor Author

@ivan-kocienski-gfsc It looks like they're describing a long running installation causing one test to fail because it's lazy installing something. I don't think that's the case for us.

@ivan-kocienski-gfsc
Copy link
Contributor

So I think the problem is: some system tests have code like:

click_link "thing"
found = select2_find_element
assert_empty found

and the select2_find_element does a CSS filter on the selinium page. So it looks for a match and finds none but as the timeout hasn't been reached it waits and tries again (the page/js may not have finished doing their things). Eventually the timeout is reached and it responds with no matching elements which is the correct value. But it had to wait for that.

The solution is to either put in some filler data (instead of looking for one item and expecting zero, put in two items and only expect one) or to move the tests out of system tests and into unit tests and only verify very very basic UI elements in system tests. (they should only be testing the very high level UX)

@ivan-kocienski-gfsc
Copy link
Contributor

Is this still a problem on our CI?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Status: Done
Development

No branches or pull requests

2 participants