Migrations v2 ignore fleet agent events

[rudolf]

Summary

Ignore fleet-agent-events and tsvb-validation-telemetry saved objects from both v1 and v2 migrations.

Fixes #95617
Fixes #91869

Release notes

Users who tried the fleet agent beta in 7.10 might have hundreds of thousands of fleet-agent-events in their .kibana index. Due to the large volume of documents, upgrades to 7.12 would frequently fail for these users. In 7.12.1 we drop all fleet-agent-events during the upgrade migration to address this problem. The fleet-agent-events documents are still left in the older .kibana_N index from previous versions, once the upgrade has succeeded we recommend deleting these indices to recover the disk space, see https://www.elastic.co/guide/en/kibana/current/upgrade-migrations.html#upgrade-migrations-old-indices

Checklist

Delete any items that are not applicable to this PR.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)
• Any UI touched in this PR does not create any new axe failures (run axe in browser: FF, Chrome)
• If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
• This renders correctly on smaller devices using a responsive layout. (You can test this in your browser)
• This was checked for cross-browser compatibility

For maintainers

• This was checked for breaking API changes and was labeled appropriately

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[rudolf]

@elastic/fleet @nchaulet somehow this test is failing when I remove fleet-agent-events. The same test also fails on master if I enable v1 migrations for this test with the following:

--- a/x-pack/test/fleet_api_integration/config.ts
+++ b/x-pack/test/fleet_api_integration/config.ts
@@ -65,6 +65,7 @@ export default async function ({ readConfigFile }: FtrConfigProviderContext) {
     kbnTestServer: {
       ...xPackAPITestsConfig.get('kbnTestServer'),
       serverArgs: [
+        '--migrations.enableV2=false',
         ...xPackAPITestsConfig.get('kbnTestServer.serverArgs'),
         ...(registryPort ? [`--xpack.fleet.registryUrl=http://localhost:${registryPort}`] : []),
       ],

We're targeting 7.12.1 for this fix, so please take a look and let me know if it's possible to fix this test or if we should skip it.

[nchaulet]

I think it's okay to skip it, we are going to remove that API in 7.13.

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[afharo]

🚀 LGTM! Just added a NIT about term vs. terms. According to https://discuss.elastic.co/t/term-vs-terms-filter/15533, the only difference in performance is caching. Since, ideally, we should be running this once, I guess it makes no difference.

[rudolf]

@elasticmachine merge upstream

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 2048186
• Storybooks Preview

Metrics [docs]

✅ unchanged

History

• 💔 Build #118515 failed 2eeac2a
• 💔 Build #118504 failed 259308c

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[kibanamachine]

💔 Backport failed

Status	Branch	Result
❌	7.12	Commit could not be cherrypicked due to conflicts
✅	7.x

Successful backport PRs will be merged automatically after passing CI.

To backport manually run:
node scripts/backport --pr 96690

[tonymeehan]

👍 🥇 💯