[ftr] handle unexpected Kibana/ES shutdowns better #131767
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
spalger
added
Team:Operations
|
Pinging @elastic/kibana-operations (Team:Operations) |
jbudz
approved these changes
May 9, 2022
💚 Build SucceededMetrics [docs]Public APIs missing comments
History
To update your PR or re-run it, just comment with: |
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
spalger
pushed a commit
to spalger/kibana
that referenced
this pull request
May 9, 2022
(cherry picked from commit c6108ba) # Conflicts: # packages/kbn-test/src/functional_test_runner/functional_test_runner.ts
spalger
pushed a commit
to spalger/kibana
that referenced
this pull request
May 9, 2022
(cherry picked from commit c6108ba) # Conflicts: # packages/kbn-es/src/cluster.js # packages/kbn-test/src/es/test_es_cluster.ts # packages/kbn-test/src/functional_test_runner/functional_test_runner.ts # packages/kbn-test/src/functional_tests/lib/run_elasticsearch.ts # packages/kbn-test/src/functional_tests/tasks.ts # packages/kbn-typed-react-router-config/src/route.ts
spalger
pushed a commit
that referenced
this pull request
May 9, 2022
kertal
pushed a commit
to kertal/kibana
that referenced
this pull request
May 24, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #131522
Adds the ability for the FTR to be notified when ProcRunner or ES Cluster instances exit unexpectedly in the background via a new
onEarlyExit()callback passed to these two classes. They both work similarly in that they return promises for astart()function, and then after the tests return a promise forstop(), but in between when the process they monitor exits they both behave differently. ES Cluster's log about the exit, and ProcRunner instances reject a promise which can't be listened to, causing the process to exit. This leads to the problem described in #131522 because when the process it exitted the test runner doesn't get a chance to write the report and devs can't inspect the reasoning for the failure. This changes the behavior to instead inject an error into mocha viarunner.uncaught(error)which treats the passed error as an error caused by the current runnable under tests, thenrunner.abort()is called to stop the tests. This isn't a very clean way to exit, but it still provides the reporting we need to understand what happened.Confirmed with https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/570#_ that Kibana failures which kill the process properly kill the job and trigger a useful report of the test that was running when the failure occurred: https://buildkite.com/organizations/elastic/pipelines/kibana-flaky-test-suite-runner/builds/570/jobs/9e3dbed4-bab9-4513-a9d1-ab372415fdae/artifacts/b5a9f677-8ac8-4f04-9bcd-c989769d4488