[esArchiver] update security solution api tests to not override Saved Object indexes #168973
Labels
Comments
dmlemeshko
added
Team:Security Solution Platform
dmlemeshko
added a commit
that referenced
this issue
Oct 27, 2023
## Summary Related to #161882 The goal is to prevent FTR tests from new esArchives that overrides SO indexes. This PR adds the existing archives that re-create Saved Objects indexes into temporary exception list, located in `packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json`. Whenever tests/archives are updated to not modify SO indexes, archive is expected to be removed from the list (progress can be tracked in #169075, #168973, #168969, #168926 ) Load action has a check if index is SO index and if the archive is in the exception list. This will throw error for the new archives, but still work as usual for the existing ones while teams updating the tests. Whenever test is loading archive listed in the exception list, the following warning message is logged: ``` warn x-pack/test/functional/es_archives/data/search_sessions overrides Saved Objects index(es) and placed temporary in the exception list. Please fix the archive and remove it from /Users/dmle/github/kibana/packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json. For more details see: #161882 ``` If the test loads a newly added archive that modifies a Saved Object index (e.g. `.kibana`), esArchiver will throw the error: ``` Error: esArchiver doesn't support modifying the existing Saved Objects index: '.kibana_1', please update its definition in mappings.json ```
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Oct 27, 2023
…69852) ## Summary Related to elastic#161882 The goal is to prevent FTR tests from new esArchives that overrides SO indexes. This PR adds the existing archives that re-create Saved Objects indexes into temporary exception list, located in `packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json`. Whenever tests/archives are updated to not modify SO indexes, archive is expected to be removed from the list (progress can be tracked in elastic#169075, elastic#168973, elastic#168969, elastic#168926 ) Load action has a check if index is SO index and if the archive is in the exception list. This will throw error for the new archives, but still work as usual for the existing ones while teams updating the tests. Whenever test is loading archive listed in the exception list, the following warning message is logged: ``` warn x-pack/test/functional/es_archives/data/search_sessions overrides Saved Objects index(es) and placed temporary in the exception list. Please fix the archive and remove it from /Users/dmle/github/kibana/packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json. For more details see: elastic#161882 ``` If the test loads a newly added archive that modifies a Saved Object index (e.g. `.kibana`), esArchiver will throw the error: ``` Error: esArchiver doesn't support modifying the existing Saved Objects index: '.kibana_1', please update its definition in mappings.json ``` (cherry picked from commit 6698810)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Oct 27, 2023
…69852) ## Summary Related to elastic#161882 The goal is to prevent FTR tests from new esArchives that overrides SO indexes. This PR adds the existing archives that re-create Saved Objects indexes into temporary exception list, located in `packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json`. Whenever tests/archives are updated to not modify SO indexes, archive is expected to be removed from the list (progress can be tracked in elastic#169075, elastic#168973, elastic#168969, elastic#168926 ) Load action has a check if index is SO index and if the archive is in the exception list. This will throw error for the new archives, but still work as usual for the existing ones while teams updating the tests. Whenever test is loading archive listed in the exception list, the following warning message is logged: ``` warn x-pack/test/functional/es_archives/data/search_sessions overrides Saved Objects index(es) and placed temporary in the exception list. Please fix the archive and remove it from /Users/dmle/github/kibana/packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json. For more details see: elastic#161882 ``` If the test loads a newly added archive that modifies a Saved Object index (e.g. `.kibana`), esArchiver will throw the error: ``` Error: esArchiver doesn't support modifying the existing Saved Objects index: '.kibana_1', please update its definition in mappings.json ``` (cherry picked from commit 6698810)
kibanamachine
added a commit
that referenced
this issue
Oct 27, 2023
…69852) (#170065) # Backport This will backport the following commits from `main` to `8.11`: - [[esArchiver] restrict from modifying saved objects indexes (#169852)](#169852) <!--- Backport version: 8.9.7 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Dzmitry Lemechko","email":"dzmitry.lemechko@elastic.co"},"sourceCommit":{"committedDate":"2023-10-27T18:27:10Z","message":"[esArchiver] restrict from modifying saved objects indexes (#169852)\n\n## Summary\r\n\r\nRelated to #161882\r\n\r\nThe goal is to prevent FTR tests from new esArchives that overrides SO\r\nindexes.\r\n\r\nThis PR adds the existing archives that re-create Saved Objects indexes\r\ninto temporary exception list, located in\r\n`packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json`.\r\nWhenever tests/archives are updated to not modify SO indexes, archive is\r\nexpected to be removed from the list (progress can be tracked in\r\n#169075, #168973, #168969, #168926 )\r\n\r\nLoad action has a check if index is SO index and if the archive is in\r\nthe exception list. This will throw error for the new archives, but\r\nstill work as usual for the existing ones while teams updating the\r\ntests.\r\n\r\nWhenever test is loading archive listed in the exception list, the\r\nfollowing warning message is logged:\r\n```\r\nwarn x-pack/test/functional/es_archives/data/search_sessions overrides Saved Objects index(es) and placed temporary in the exception list.\r\nPlease fix the archive and remove it from /Users/dmle/github/kibana/packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json.\r\nFor more details see: https://github.com/elastic/kibana/issues/161882\r\n```\r\n\r\nIf the test loads a newly added archive that modifies a Saved Object\r\nindex (e.g. `.kibana`), esArchiver will throw the error:\r\n```\r\n Error: esArchiver doesn't support modifying the existing Saved Objects index: '.kibana_1',\r\n please update its definition in mappings.json\r\n```","sha":"669881095881b599262d8ba19bdf68207c86bfa0","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","ES-ARCHIVER","v8.12.0","v8.11.1","v8.10.5"],"number":169852,"url":"https://github.com/elastic/kibana/pull/169852","mergeCommit":{"message":"[esArchiver] restrict from modifying saved objects indexes (#169852)\n\n## Summary\r\n\r\nRelated to #161882\r\n\r\nThe goal is to prevent FTR tests from new esArchives that overrides SO\r\nindexes.\r\n\r\nThis PR adds the existing archives that re-create Saved Objects indexes\r\ninto temporary exception list, located in\r\n`packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json`.\r\nWhenever tests/archives are updated to not modify SO indexes, archive is\r\nexpected to be removed from the list (progress can be tracked in\r\n#169075, #168973, #168969, #168926 )\r\n\r\nLoad action has a check if index is SO index and if the archive is in\r\nthe exception list. This will throw error for the new archives, but\r\nstill work as usual for the existing ones while teams updating the\r\ntests.\r\n\r\nWhenever test is loading archive listed in the exception list, the\r\nfollowing warning message is logged:\r\n```\r\nwarn x-pack/test/functional/es_archives/data/search_sessions overrides Saved Objects index(es) and placed temporary in the exception list.\r\nPlease fix the archive and remove it from /Users/dmle/github/kibana/packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json.\r\nFor more details see: https://github.com/elastic/kibana/issues/161882\r\n```\r\n\r\nIf the test loads a newly added archive that modifies a Saved Object\r\nindex (e.g. `.kibana`), esArchiver will throw the error:\r\n```\r\n Error: esArchiver doesn't support modifying the existing Saved Objects index: '.kibana_1',\r\n please update its definition in mappings.json\r\n```","sha":"669881095881b599262d8ba19bdf68207c86bfa0"}},"sourceBranch":"main","suggestedTargetBranches":["8.11","8.10"],"targetPullRequestStates":[{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/169852","number":169852,"mergeCommit":{"message":"[esArchiver] restrict from modifying saved objects indexes (#169852)\n\n## Summary\r\n\r\nRelated to #161882\r\n\r\nThe goal is to prevent FTR tests from new esArchives that overrides SO\r\nindexes.\r\n\r\nThis PR adds the existing archives that re-create Saved Objects indexes\r\ninto temporary exception list, located in\r\n`packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json`.\r\nWhenever tests/archives are updated to not modify SO indexes, archive is\r\nexpected to be removed from the list (progress can be tracked in\r\n#169075, #168973, #168969, #168926 )\r\n\r\nLoad action has a check if index is SO index and if the archive is in\r\nthe exception list. This will throw error for the new archives, but\r\nstill work as usual for the existing ones while teams updating the\r\ntests.\r\n\r\nWhenever test is loading archive listed in the exception list, the\r\nfollowing warning message is logged:\r\n```\r\nwarn x-pack/test/functional/es_archives/data/search_sessions overrides Saved Objects index(es) and placed temporary in the exception list.\r\nPlease fix the archive and remove it from /Users/dmle/github/kibana/packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json.\r\nFor more details see: https://github.com/elastic/kibana/issues/161882\r\n```\r\n\r\nIf the test loads a newly added archive that modifies a Saved Object\r\nindex (e.g. `.kibana`), esArchiver will throw the error:\r\n```\r\n Error: esArchiver doesn't support modifying the existing Saved Objects index: '.kibana_1',\r\n please update its definition in mappings.json\r\n```","sha":"669881095881b599262d8ba19bdf68207c86bfa0"}},{"branch":"8.11","label":"v8.11.1","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.10","label":"v8.10.5","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
kibanamachine
added a commit
that referenced
this issue
Oct 30, 2023
…69852) (#170064) # Backport This will backport the following commits from `main` to `8.10`: - [[esArchiver] restrict from modifying saved objects indexes (#169852)](#169852) <!--- Backport version: 8.9.7 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Dzmitry Lemechko","email":"dzmitry.lemechko@elastic.co"},"sourceCommit":{"committedDate":"2023-10-27T18:27:10Z","message":"[esArchiver] restrict from modifying saved objects indexes (#169852)\n\n## Summary\r\n\r\nRelated to #161882\r\n\r\nThe goal is to prevent FTR tests from new esArchives that overrides SO\r\nindexes.\r\n\r\nThis PR adds the existing archives that re-create Saved Objects indexes\r\ninto temporary exception list, located in\r\n`packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json`.\r\nWhenever tests/archives are updated to not modify SO indexes, archive is\r\nexpected to be removed from the list (progress can be tracked in\r\n#169075, #168973, #168969, #168926 )\r\n\r\nLoad action has a check if index is SO index and if the archive is in\r\nthe exception list. This will throw error for the new archives, but\r\nstill work as usual for the existing ones while teams updating the\r\ntests.\r\n\r\nWhenever test is loading archive listed in the exception list, the\r\nfollowing warning message is logged:\r\n```\r\nwarn x-pack/test/functional/es_archives/data/search_sessions overrides Saved Objects index(es) and placed temporary in the exception list.\r\nPlease fix the archive and remove it from /Users/dmle/github/kibana/packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json.\r\nFor more details see: https://github.com/elastic/kibana/issues/161882\r\n```\r\n\r\nIf the test loads a newly added archive that modifies a Saved Object\r\nindex (e.g. `.kibana`), esArchiver will throw the error:\r\n```\r\n Error: esArchiver doesn't support modifying the existing Saved Objects index: '.kibana_1',\r\n please update its definition in mappings.json\r\n```","sha":"669881095881b599262d8ba19bdf68207c86bfa0","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","ES-ARCHIVER","v8.12.0","v8.11.1","v8.10.5"],"number":169852,"url":"https://github.com/elastic/kibana/pull/169852","mergeCommit":{"message":"[esArchiver] restrict from modifying saved objects indexes (#169852)\n\n## Summary\r\n\r\nRelated to #161882\r\n\r\nThe goal is to prevent FTR tests from new esArchives that overrides SO\r\nindexes.\r\n\r\nThis PR adds the existing archives that re-create Saved Objects indexes\r\ninto temporary exception list, located in\r\n`packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json`.\r\nWhenever tests/archives are updated to not modify SO indexes, archive is\r\nexpected to be removed from the list (progress can be tracked in\r\n#169075, #168973, #168969, #168926 )\r\n\r\nLoad action has a check if index is SO index and if the archive is in\r\nthe exception list. This will throw error for the new archives, but\r\nstill work as usual for the existing ones while teams updating the\r\ntests.\r\n\r\nWhenever test is loading archive listed in the exception list, the\r\nfollowing warning message is logged:\r\n```\r\nwarn x-pack/test/functional/es_archives/data/search_sessions overrides Saved Objects index(es) and placed temporary in the exception list.\r\nPlease fix the archive and remove it from /Users/dmle/github/kibana/packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json.\r\nFor more details see: https://github.com/elastic/kibana/issues/161882\r\n```\r\n\r\nIf the test loads a newly added archive that modifies a Saved Object\r\nindex (e.g. `.kibana`), esArchiver will throw the error:\r\n```\r\n Error: esArchiver doesn't support modifying the existing Saved Objects index: '.kibana_1',\r\n please update its definition in mappings.json\r\n```","sha":"669881095881b599262d8ba19bdf68207c86bfa0"}},"sourceBranch":"main","suggestedTargetBranches":["8.11","8.10"],"targetPullRequestStates":[{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/169852","number":169852,"mergeCommit":{"message":"[esArchiver] restrict from modifying saved objects indexes (#169852)\n\n## Summary\r\n\r\nRelated to #161882\r\n\r\nThe goal is to prevent FTR tests from new esArchives that overrides SO\r\nindexes.\r\n\r\nThis PR adds the existing archives that re-create Saved Objects indexes\r\ninto temporary exception list, located in\r\n`packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json`.\r\nWhenever tests/archives are updated to not modify SO indexes, archive is\r\nexpected to be removed from the list (progress can be tracked in\r\n#169075, #168973, #168969, #168926 )\r\n\r\nLoad action has a check if index is SO index and if the archive is in\r\nthe exception list. This will throw error for the new archives, but\r\nstill work as usual for the existing ones while teams updating the\r\ntests.\r\n\r\nWhenever test is loading archive listed in the exception list, the\r\nfollowing warning message is logged:\r\n```\r\nwarn x-pack/test/functional/es_archives/data/search_sessions overrides Saved Objects index(es) and placed temporary in the exception list.\r\nPlease fix the archive and remove it from /Users/dmle/github/kibana/packages/kbn-es-archiver/src/fixtures/override_saved_objects_index/exception_list.json.\r\nFor more details see: https://github.com/elastic/kibana/issues/161882\r\n```\r\n\r\nIf the test loads a newly added archive that modifies a Saved Object\r\nindex (e.g. `.kibana`), esArchiver will throw the error:\r\n```\r\n Error: esArchiver doesn't support modifying the existing Saved Objects index: '.kibana_1',\r\n please update its definition in mappings.json\r\n```","sha":"669881095881b599262d8ba19bdf68207c86bfa0"}},{"branch":"8.11","label":"v8.11.1","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.10","label":"v8.10.5","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The problem:
tl;dr; esArchiver will restrict modifying Saved Object (SO) indexes soon, this change will break some existing tests.
The reason: esArchiver has been here for years, a great tool to upload test data in Elasticsearch.
It allows defining archives that contain both ES indices definitions (mappings.json) and documents to be imported in those indices (data.json).
While this is totally fine for data indexes, it currently also allows defining and overriding Saved Object (SO) indices.
SO indices are created and managed by Kibana internally.
At startup, Kibana ensures that SO indices and their mappings are aligned with the current stack version.
Through esArchiver, some tests are currently:
This is less than ideal, it brings unnecessary complexity (migrations) and flakiness, and it enables tests to be run against system indices that are not necessarily aligned with current stack version.
Requested change:
Ideally the goal is to
mappings.jsonfiles.data.json.Example #162321
The following tests re-create
.kibanaindex and should be updated:We also suggest to convert
x-pack/test/api_integration/apis/security_solution/timeline_migrations.tsinto jest integration test, see more details in kibana/issues/168926#issuecomment-1764284188The text was updated successfully, but these errors were encountered: