Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[esArchiver] update reporting & search session api tests to not override Saved Object indexes #169075

Closed
Tracked by #161882
dmlemeshko opened this issue Oct 17, 2023 · 1 comment · Fixed by #174077
Closed
Tracked by #161882

[esArchiver] update reporting & search session api tests to not override Saved Object indexes #169075

dmlemeshko opened this issue Oct 17, 2023 · 1 comment · Fixed by #174077
Assignees
@vadimkibana
Labels
ES-ARCHIVER Team:SharedUX Team label for AppEx-SharedUX (formerly Global Experience) test-api-integration

Comments

@dmlemeshko
Copy link
Member

The problem:

tl;dr; esArchiver will restrict modifying Saved Object (SO) indexes soon, this change will break some existing tests.

The reason: esArchiver has been here for years, a great tool to upload test data in Elasticsearch.
It allows defining archives that contain both ES indices definitions (mappings.json) and documents to be imported in those indices (data.json).
While this is totally fine for data indexes, it currently also allows defining and overriding Saved Object (SO) indices.
SO indices are created and managed by Kibana internally.
At startup, Kibana ensures that SO indices and their mappings are aligned with the current stack version.

Through esArchiver, some tests are currently:

  • deleting SO indices while Kibana is already running
  • re-creating those SO indices with static mappings (that are not necessarily aligned with current stack version)

This is less than ideal, it brings unnecessary complexity (migrations) and flakiness, and it enables tests to be run against system indices that are not necessarily aligned with current stack version.

Requested change:
Ideally the goal is to

  • remove all SO index definitions from all mappings.json files.
  • update documents defined in data.json.

Example #162321

The following tests re-create .kibana index and should be updated:

  • x-pack/test/reporting_functional/reporting_and_security/security_roles_privileges.ts
  • x-pack/test/search_sessions_integration/tests/apps/management/search_sessions/sessions_management.ts
  • x-pack/test/reporting_functional/reporting_and_deprecated_security/security_roles_privileges.ts
@botelastic botelastic bot added the needs-team Issues missing a team label label Oct 17, 2023
@dmlemeshko dmlemeshko added test-api-integration Team:SharedUX Team label for AppEx-SharedUX (formerly Global Experience) ES-ARCHIVER and removed needs-team Issues missing a team label labels Oct 17, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/appex-sharedux (Team:SharedUX)

@rshen91 rshen91 mentioned this issue Oct 23, 2023
Closed
2 tasks
@dmlemeshko dmlemeshko mentioned this issue Oct 26, 2023
Merged
dmlemeshko added a commit that referenced this issue Oct 27, 2023
@dmlemeshko
[esArchiver] restrict from modifying saved objects indexes (#169852)
6698810 
## Summary

Related to #161882

The goal is to prevent FTR tests from new esArchives that overrides SO
indexes.

This PR adds the existing archives that re-create Saved Objects indexes
into temporary exception list, located in
`packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json`.
Whenever tests/archives are updated to not modify SO indexes, archive is
expected to be removed from the list (progress can be tracked in
#169075, #168973, #168969, #168926 )

Load action has a check if index is SO index and if the archive is in
the exception list. This will throw error for the new archives, but
still work as usual for the existing ones while teams updating the
tests.

Whenever test is loading archive listed in the exception list, the
following warning message is logged:
```
warn x-pack/test/functional/es_archives/data/search_sessions overrides Saved Objects index(es) and placed temporary in the exception list.
Please fix the archive and remove it from /Users/dmle/github/kibana/packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json.
For more details see: #161882
```

If the test loads a newly added archive that modifies a Saved Object
index (e.g. `.kibana`), esArchiver will throw the error:
```
 Error: esArchiver doesn't support modifying the existing Saved Objects index: '.kibana_1',
 please update its definition in mappings.json
```
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Oct 27, 2023
@dmlemeshko
[esArchiver] restrict from modifying saved objects indexes (elastic#1…
8a36813 
…69852)

## Summary

Related to elastic#161882

The goal is to prevent FTR tests from new esArchives that overrides SO
indexes.

This PR adds the existing archives that re-create Saved Objects indexes
into temporary exception list, located in
`packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json`.
Whenever tests/archives are updated to not modify SO indexes, archive is
expected to be removed from the list (progress can be tracked in
elastic#169075, elastic#168973, elastic#168969, elastic#168926 )

Load action has a check if index is SO index and if the archive is in
the exception list. This will throw error for the new archives, but
still work as usual for the existing ones while teams updating the
tests.

Whenever test is loading archive listed in the exception list, the
following warning message is logged:
```
warn x-pack/test/functional/es_archives/data/search_sessions overrides Saved Objects index(es) and placed temporary in the exception list.
Please fix the archive and remove it from /Users/dmle/github/kibana/packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json.
For more details see: elastic#161882
```

If the test loads a newly added archive that modifies a Saved Object
index (e.g. `.kibana`), esArchiver will throw the error:
```
 Error: esArchiver doesn't support modifying the existing Saved Objects index: '.kibana_1',
 please update its definition in mappings.json
```

(cherry picked from commit 6698810)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Oct 27, 2023
@dmlemeshko
[esArchiver] restrict from modifying saved objects indexes (elastic#1…
36ccc83 
…69852)

## Summary

Related to elastic#161882

The goal is to prevent FTR tests from new esArchives that overrides SO
indexes.

This PR adds the existing archives that re-create Saved Objects indexes
into temporary exception list, located in
`packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json`.
Whenever tests/archives are updated to not modify SO indexes, archive is
expected to be removed from the list (progress can be tracked in
elastic#169075, elastic#168973, elastic#168969, elastic#168926 )

Load action has a check if index is SO index and if the archive is in
the exception list. This will throw error for the new archives, but
still work as usual for the existing ones while teams updating the
tests.

Whenever test is loading archive listed in the exception list, the
following warning message is logged:
```
warn x-pack/test/functional/es_archives/data/search_sessions overrides Saved Objects index(es) and placed temporary in the exception list.
Please fix the archive and remove it from /Users/dmle/github/kibana/packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json.
For more details see: elastic#161882
```

If the test loads a newly added archive that modifies a Saved Object
index (e.g. `.kibana`), esArchiver will throw the error:
```
 Error: esArchiver doesn't support modifying the existing Saved Objects index: '.kibana_1',
 please update its definition in mappings.json
```

(cherry picked from commit 6698810)
@dmlemeshko dmlemeshko mentioned this issue Nov 29, 2023
Open
@vadimkibana vadimkibana mentioned this issue Jan 2, 2024
Merged
vadimkibana added a commit that referenced this issue Mar 8, 2024
@vadimkibana @kibanamachine
[Reporting] Remove reporting tests that depend on esArchiver (#174077)
1056f2e 
## Summary

Closes #169075

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Mar 8, 2024
@vadimkibana
[Reporting] Remove reporting tests that depend on esArchiver (elastic…
fda05dc 
…#174077)

## Summary

Closes elastic#169075

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit 1056f2e)
kibanamachine referenced this issue Mar 8, 2024
@kibanamachine @vadimkibana
[8.13] [Reporting] Remove reporting tests that depend on esArchiver (#…
59be28e 
…174077) (#178296)

# Backport

This will backport the following commits from `main` to `8.13`:
- [[Reporting] Remove reporting tests that depend on esArchiver
(#174077)](#174077)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Vadim
Kibana","email":"82822460+vadimkibana@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-03-08T11:22:31Z","message":"[Reporting]
Remove reporting tests that depend on esArchiver (#174077)\n\n##
Summary\r\n\r\nCloses
https://github.com/elastic/kibana/issues/169075\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"1056f2ed8f9deefca1855ea60a85900c6915ac7d","branchLabelMapping":{"^v8.14.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["review","release_note:skip","Team:SharedUX","v8.13.0","v8.14.0"],"title":"[Reporting]
Remove reporting tests that depend on
esArchiver","number":174077,"url":"https://github.com/elastic/kibana/pull/174077","mergeCommit":{"message":"[Reporting]
Remove reporting tests that depend on esArchiver (#174077)\n\n##
Summary\r\n\r\nCloses
https://github.com/elastic/kibana/issues/169075\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"1056f2ed8f9deefca1855ea60a85900c6915ac7d"}},"sourceBranch":"main","suggestedTargetBranches":["8.13"],"targetPullRequestStates":[{"branch":"8.13","label":"v8.13.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.14.0","branchLabelMappingKey":"^v8.14.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/174077","number":174077,"mergeCommit":{"message":"[Reporting]
Remove reporting tests that depend on esArchiver (#174077)\n\n##
Summary\r\n\r\nCloses
https://github.com/elastic/kibana/issues/169075\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"1056f2ed8f9deefca1855ea60a85900c6915ac7d"}}]}]
BACKPORT-->

Co-authored-by: Vadim Kibana <82822460+vadimkibana@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vadimkibana vadimkibana

Labels
ES-ARCHIVER Team:SharedUX Team label for AppEx-SharedUX (formerly Global Experience) test-api-integration
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Reporting] Remove reporting tests that depend on esArchiver vadimkibana/kibana
3 participants
@dmlemeshko @elasticmachine @vadimkibana