Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Data Discovery] Unskip and fix Serverless API integration tests #168116

Merged
merged 3 commits into from
Oct 6, 2023
Merged

[Data Discovery] Unskip and fix Serverless API integration tests #168116

merged 3 commits into from
Oct 6, 2023

Conversation

davismcphee
Copy link
Contributor

@davismcphee davismcphee commented Oct 5, 2023 •
edited by kibanamachine
Loading

Summary

This PR unskips and fixes previously flaky Serverless API integration tests.

Flaky test runs:

Resolves #165868.
Resolves #165944.
Resolves #165972.
Resolves #165942.
Resolves #165476.
Resolves #165511.

Checklist

For maintainers

@davismcphee davismcphee added release_note:skip Skip the PR/issue when compiling release notes Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. v8.11.0 v8.12.0 labels Oct 5, 2023
@davismcphee davismcphee self-assigned this Oct 5, 2023
@davismcphee davismcphee force-pushed the unksip-serverless-api-tests branch from 0a91275 to cbbb152 Compare October 5, 2023 15:40
davismcphee
davismcphee commented Oct 5, 2023
View reviewed changes
...api_integration/test_suites/common/data_views/default_index_pattern/default_index_pattern.ts
Comment on lines +72 to +75
// TODO: The response comes back undefined in Serverless
const body = response6.body[serviceKeyId];
const expected = body === undefined ? null : body;
expect(expected).to.be(null);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattkime do you think this is ok? Clearing the default data view works in Serverless by passing null, but the return value after fetching is now undefined instead of null as it is in stateful.

@davismcphee davismcphee marked this pull request as ready for review October 5, 2023 17:35
@davismcphee davismcphee requested a review from a team as a code owner October 5, 2023 17:35
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @davismcphee

@davismcphee davismcphee merged commit c95721a into elastic:main Oct 6, 2023
@davismcphee davismcphee deleted the unksip-serverless-api-tests branch October 6, 2023 18:32
@kibanamachine kibanamachine mentioned this pull request Oct 6, 2023
Merged
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 6, 2023
@davismcphee
[Data Discovery] Unskip and fix Serverless API integration tests (ela…
73dadc8 
…stic#168116)

## Summary

This PR unskips and fixes previously flaky Serverless API integration
tests.

Flaky test runs:
- x100:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3364

Resolves elastic#165868.
Resolves elastic#165944.
Resolves elastic#165972.
Resolves elastic#165942.
Resolves elastic#165476.
Resolves elastic#165511.

### Checklist

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

(cherry picked from commit c95721a)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
✅ 8.11

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Oct 6, 2023
@kibanamachine @davismcphee
[8.11] [Data Discovery] Unskip and fix Serverless API integration tes…
6c1ea44 
…ts (#168116) (#168289)

# Backport

This will backport the following commits from `main` to `8.11`:
- [[Data Discovery] Unskip and fix Serverless API integration tests
(#168116)](#168116)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Davis
McPhee","email":"davis.mcphee@elastic.co"},"sourceCommit":{"committedDate":"2023-10-06T18:32:29Z","message":"[Data
Discovery] Unskip and fix Serverless API integration tests
(#168116)\n\n## Summary\r\n\r\nThis PR unskips and fixes previously
flaky Serverless API integration\r\ntests.\r\n\r\nFlaky test runs:\r\n-
x100:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3364\r\n\r\nResolves
#165868.\r\nResolves #165944.\r\nResolves #165972.\r\nResolves
#165942.\r\nResolves #165476.\r\nResolves #165511.\r\n\r\n###
Checklist\r\n\r\n- [ ] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [x] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] Any UI
touched in this PR is usable by keyboard only (learn more\r\nabout
[keyboard accessibility](https://webaim.org/techniques/keyboard/))\r\n-
[ ] Any UI touched in this PR does not create any new axe
failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n###
For maintainers\r\n\r\n- [ ] This was checked for breaking API changes
and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"c95721a40b25ab583e6f180ce5f857a6532e56e6","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:DataDiscovery","v8.11.0","v8.12.0"],"number":168116,"url":"https://github.com/elastic/kibana/pull/168116","mergeCommit":{"message":"[Data
Discovery] Unskip and fix Serverless API integration tests
(#168116)\n\n## Summary\r\n\r\nThis PR unskips and fixes previously
flaky Serverless API integration\r\ntests.\r\n\r\nFlaky test runs:\r\n-
x100:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3364\r\n\r\nResolves
#165868.\r\nResolves #165944.\r\nResolves #165972.\r\nResolves
#165942.\r\nResolves #165476.\r\nResolves #165511.\r\n\r\n###
Checklist\r\n\r\n- [ ] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [x] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] Any UI
touched in this PR is usable by keyboard only (learn more\r\nabout
[keyboard accessibility](https://webaim.org/techniques/keyboard/))\r\n-
[ ] Any UI touched in this PR does not create any new axe
failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n###
For maintainers\r\n\r\n- [ ] This was checked for breaking API changes
and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"c95721a40b25ab583e6f180ce5f857a6532e56e6"}},"sourceBranch":"main","suggestedTargetBranches":["8.11"],"targetPullRequestStates":[{"branch":"8.11","label":"v8.11.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/168116","number":168116,"mergeCommit":{"message":"[Data
Discovery] Unskip and fix Serverless API integration tests
(#168116)\n\n## Summary\r\n\r\nThis PR unskips and fixes previously
flaky Serverless API integration\r\ntests.\r\n\r\nFlaky test runs:\r\n-
x100:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3364\r\n\r\nResolves
#165868.\r\nResolves #165944.\r\nResolves #165972.\r\nResolves
#165942.\r\nResolves #165476.\r\nResolves #165511.\r\n\r\n###
Checklist\r\n\r\n- [ ] Any text added follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [x] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] Any UI
touched in this PR is usable by keyboard only (learn more\r\nabout
[keyboard accessibility](https://webaim.org/techniques/keyboard/))\r\n-
[ ] Any UI touched in this PR does not create any new axe
failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n###
For maintainers\r\n\r\n- [ ] This was checked for breaking API changes
and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"c95721a40b25ab583e6f180ce5f857a6532e56e6"}}]}]
BACKPORT-->

Co-authored-by: Davis McPhee <davis.mcphee@elastic.co>
dej611 pushed a commit to dej611/kibana that referenced this pull request Oct 17, 2023
@davismcphee @dej611
[Data Discovery] Unskip and fix Serverless API integration tests (ela…
76a05c1 
…stic#168116)

## Summary

This PR unskips and fixes previously flaky Serverless API integration
tests.

Flaky test runs:
- x100:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3364

Resolves elastic#165868.
Resolves elastic#165944.
Resolves elastic#165972.
Resolves elastic#165942.
Resolves elastic#165476.
Resolves elastic#165511.

### Checklist

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jughosta jughosta jughosta approved these changes

Assignees

@davismcphee davismcphee

Labels
release_note:skip Skip the PR/issue when compiling release notes Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. v8.11.0 v8.12.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Failing test: Serverless Observability API Integration Tests.x-pack/test_serverless/api_integration/test_suites/common/data_views/es_errors/errors·js - serverless common API index_patterns index_patterns/service/lib index_patterns/* error handler "before all" hook in "index_patterns/* error handler" Failing test: Serverless Observability API Integration Tests.x-pack/test_serverless/api_integration/test_suites/common/data_views/fields_for_wildcard_route/params·ts - serverless common API index_patterns index_patterns/_fields_for_wildcard route params "before all" hook for "requires a pattern query param" Failing test: Serverless Observability API Integration Tests.x-pack/test_serverless/api_integration/test_suites/common/data_view_field_editor/field_preview·ts - serverless common API index pattern field editor Field preview "before all" hook in "Field preview" Failing test: Serverless Observability API Integration Tests.x-pack/test_serverless/api_integration/test_suites/common/scripted_fields·ts - serverless common API scripted fields disabled "before all" hook for "scripted fields are ignored when disabled" Failing test: Serverless Observability API Integration Tests.x-pack/test_serverless/api_integration/test_suites/common/rollups·ts - serverless common API rollup data views - fields for wildcard "before all" hook for "returns 200 and best effort response despite lack of rollup support"
Failing test: Serverless Observability API Integration Tests.x-pack/test_serverless/api_integration/test_suites/common/data_views/fields_for_wildcard_route/conflicts·ts - serverless common API index_patterns index_patterns/_fields_for_wildcard route conflicts "before all" hook for "flags fields with mismatched types as conflicting"
5 participants
@davismcphee @elasticmachine @kibana-ci @kibanamachine @jughosta