Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve x-pack-ciGroup6 tests stability with WebDriver #31454

Merged
merged 1 commit into from
Feb 26, 2019
Merged

Improve x-pack-ciGroup6 tests stability with WebDriver #31454

merged 1 commit into from
Feb 26, 2019

Conversation

dmlemeshko
Copy link
Member

@elasticmachine
Copy link
Contributor

💔 Build Failed

@dmlemeshko
Copy link
Member Author

dmlemeshko commented Feb 19, 2019 •
edited
Loading

jobs elastic kibana pull-request job x-pack-cigrou

X-Pack Chromium Functional Reporting Tests.x-pack/test/reporting/functional/reporting·js.reporting app Reporting Dashboard Print PNG button becomes available when saved

 Error: expected testSubject(saveDashboardSuccess) to exist
    at TestSubjects.existOrFail (/var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup6-11/node/immutable/kibana/test/functional/services/test_subjects.js:53:15)
    at process._tickCallback (internal/process/next_tick.js:68:7)

On screenshot, we can see save dialog is displayed (shadowed). Will try to dig in.

@elasticmachine
Copy link
Contributor

💔 Build Failed

@dmlemeshko
Copy link
Member Author

The only failure is:
X-Pack API Integration Tests.x-pack/test/api_integration/apis/beats/remove_tags_from_beats·js.apis beats remove_tags_from_beats should return errors for non-existent beats

I guess its not related. Will rerun jobs later to check flakiness

@dmlemeshko
Copy link
Member Author

retest

@elasticmachine
Copy link
Contributor

💔 Build Failed

@dmlemeshko
Copy link
Member Author

dmlemeshko commented Feb 19, 2019 •
edited
Loading

This time it is:

X-Pack Chromium Functional Reporting Tests.x-pack/test/reporting/functional/reporting·js.reporting app Reporting Discover Generate CSV button generates a report with no data

@dmlemeshko
Copy link
Member Author

retest

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@dmlemeshko
Copy link
Member Author

retest

@elasticmachine
Copy link
Contributor

💔 Build Failed

@dmlemeshko
Copy link
Member Author

dmlemeshko commented Feb 20, 2019 •
edited
Loading

1 failure:
X-Pack Chromium Functional Reporting Tests.x-pack/test/reporting/functional/reporting·js.reporting app Reporting Visualize Print PDF button becomes available when saved

Means initial fix did not solve it completely.

reporting app reporting visualize print pdf button becomes available when saved

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@elasticmachine
Copy link
Contributor

💔 Build Failed

@dmlemeshko
Copy link
Member Author

retest

@elasticmachine
Copy link
Contributor

💔 Build Failed

@dmlemeshko
Copy link
Member Author

1 Failure 2 times in a row:
X-Pack Chromium Functional Reporting Tests.x-pack/test/reporting/functional/reporting·js.reporting app Reporting Visualize Print PDF button becomes available when saved

@dmlemeshko
Copy link
Member Author

retest

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@dmlemeshko
Copy link
Member Author

retest

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@dmlemeshko
Copy link
Member Author

retest

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@dmlemeshko dmlemeshko force-pushed the implement/ftr-webdriver-x-pack-ciGroup6 branch from 72744d5 to 34e7021 Compare February 25, 2019 09:33
@dmlemeshko dmlemeshko changed the title [page_objects/visualize_page] open modal dialog only 1 time Improve x-pack-ciGroup6 tests stability with WebDriver Feb 25, 2019
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@dmlemeshko
Copy link
Member Author

retest

@elasticmachine
Copy link
Contributor

💔 Build Failed

@dmlemeshko dmlemeshko requested a review from spalger February 25, 2019 12:36
@dmlemeshko
Copy link
Member Author

@spalger I think with these changes reporting tests are finally stable.

@dmlemeshko
Copy link
Member Author

Unrelated failure:

11:57:02 [5/5] Building fresh packages...
11:59:19 error /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup6-1/node/immutable/kibana/node_modules/chromedriver: Command failed.
11:59:19 Exit code: 1

@dmlemeshko
Copy link
Member Author

retest

1 similar comment
@cuff-links
Copy link
Contributor

retest

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@cuff-links
Copy link
Contributor

retest

@cuff-links
Copy link
Contributor

This has worked in CI but does fail intermittently for me when I use TEST_THROTTLE_NETWORK. I tried it 20 times and it failed twice. Same test. generates a report with no data.

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@dmlemeshko dmlemeshko merged commit 098777a into elastic:implement/ftr-webdriver-x-pack-ciGroup6 Feb 26, 2019
@dmlemeshko
Copy link
Member Author

@silne30 thank you for the feedback, I'm totally fine to add more commits to fix that.

spalger pushed a commit that referenced this pull request Feb 26, 2019
[#31446] stabilize x-pack-ciGroup6 (#31447)
a823c9b 
* rerun x-pack-ciGroup6 40 times

* improve reporting tests stability (#31454)
spalger pushed a commit that referenced this pull request Mar 2, 2019
Restore webdriver-based functional test runner driver (#31446)
a6afcc5 
* Restore webdriver-based functional test runner driver

* run all function test suites 6 times per build, to help spot flakiness

* [tests/reporting] fix flaky click on report generation

* [#31446] stabilize x-pack-ciGroup6 (#31447)

* rerun x-pack-ciGroup6 40 times

* improve reporting tests stability (#31454)

* Revert "rerun x-pack-ciGroup6 40 times"

This reverts commit 63026b1.

* Fix Dashboard Save tests (#32112)

* [page_objects/dashboard_page] add waitDialogIsClosed to manage dashboard popup behaviour

* [page_objects/dashboard_page] add waitDialogIsClosed flag

* services: leadfoot -> webdriver

* [services/lib/web_element_wrapper] extend type with charByChar input (#32305)

* Revert "run all function test suites 6 times per build, to help spot flakiness"

This reverts commit 4770993.
@dmlemeshko dmlemeshko deleted the implement/ftr-webdriver-x-pack-ciGroup6 branch April 22, 2019 07:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@spalger spalger Awaiting requested review from spalger

@cuff-links cuff-links Awaiting requested review from cuff-links

Assignees
No one assigned
Labels
review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@dmlemeshko @elasticmachine @cuff-links