[Fleet] Remove legacy component templates on package install

[hop-dev]

Summary

Closes #129662

Remove any legacy @mappings or @settings component templates during package install. Only delete if:

• they are not used by any index templates ( e.g if (for some reason :D ) the user has created an index template referring to the fleet component templates)
• _meta.package.name matches the package being installed (incase the user has created their own version, or another package had created the component template, not likely but I put the check there just in case)

Background

In #121184 we introduced the @package component template which is a combination of @mappings and @settings.

In #129662 @nchaulet noticed that when installing a package in 8.2 which was previously installed in a lower version, we failed to clean up the legacy @mapping component templates
When
When installing a package, check to see if there are any legacy templates

Testing

I've added unit and functional tests.

Manual test steps:

Expand for Kibana Dev Tools Commands for Manual Testing

# create legacy mappings component template
PUT _component_template/logs-apm.app@mappings
{
  "template": {
    "mappings": {
      "dynamic": false,
      "dynamic_templates": [
        {
          "numeric_labels": {
            "path_match": "numeric_labels.*",
            "mapping": {
              "scaling_factor": 1000000,
              "type": "scaled_float"
            }
          }
        }
      ]
    }
  },
  "_meta": {
    "package": {
      "name": "apm"
    }
  }
}

# create legacy settings component template
PUT _component_template/logs-apm.app@settings
{
  "template": {
    "settings": {
      "index": {
        "lifecycle": {
          "name": "logs-apm.app_logs-default_policy"
        },
        "codec": "best_compression",
        "mapping": {
          "total_fields": {
            "limit": "10000"
          }
        },
        "query": {
          "default_field": [
            "agent.ephemeral_id",
            "agent.name",
            "agent.version",
            "client.domain",
            "client.geo.city_name",
            "client.geo.continent_name",
            "client.geo.country_iso_code",
            "client.geo.country_name",
            "client.geo.region_iso_code",
            "client.geo.region_name",
            "cloud.account.id",
            "cloud.account.name",
            "cloud.availability_zone",
            "cloud.instance.id",
            "cloud.instance.name",
            "cloud.machine.type",
            "cloud.project.id",
            "cloud.project.name",
            "cloud.provider",
            "cloud.region",
            "cloud.service.name",
            "container.id",
            "destination.address",
            "ecs.version",
            "event.outcome",
            "event.action",
            "host.architecture",
            "host.hostname",
            "host.name",
            "host.os.platform",
            "http.request.method",
            "http.request.referrer",
            "http.version",
            "log.level",
            "observer.hostname",
            "observer.name",
            "observer.type",
            "observer.version",
            "observer.ephemeral_id",
            "observer.id",
            "process.args",
            "process.title",
            "service.environment",
            "service.name",
            "service.node.name",
            "service.version",
            "service.framework.name",
            "service.framework.version",
            "service.language.name",
            "service.language.version",
            "service.runtime.name",
            "service.runtime.version",
            "source.domain",
            "span.id",
            "trace.id",
            "transaction.id",
            "url.domain",
            "url.fragment",
            "url.query",
            "url.scheme",
            "user.domain",
            "user.email",
            "user.id",
            "user.name",
            "user_agent.device.name",
            "user_agent.name",
            "user_agent.original",
            "user_agent.os.family",
            "user_agent.os.full",
            "user_agent.os.kernel",
            "user_agent.os.name",
            "user_agent.os.platform",
            "user_agent.os.version",
            "user_agent.version",
            "kubernetes.namespace",
            "kubernetes.node.name",
            "kubernetes.pod.name",
            "kubernetes.pod.uid"
          ]
        }
      }
    }
  },
  "_meta": {
    "package": {
      "name": "apm"
    }
  }
}

# create index template using legacy componente templates
POST _index_template/mytemplate
{
  "composed_of" : ["logs-apm.app@mappings","logs-apm.app@settings"],
  "index_patterns" : ["idontexist"],
  "template": {
    "mappings": {
      "dynamic": false
    }
  }
}

# check if templates exist
GET /_component_template/logs-apm.app@mappings
GET /_component_template/logs-apm.app@settings
GET /_index_template/mytemplate

Scenario 1 - component templates not used by index templates

• create fake legacy components template
• install apm integration
• observe component templates have been deleted

Scenario 2 - component templates are used by an index template

• create fake legacy components template
• create index template which is composed of the component templates
• install apm integration
• observe component templates have not been deleted

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[hop-dev]

@elasticmachine merge upstream

[kpollich]

Just a couple nitpicky code things - didn't pull down the code to test quite yet. Thanks for all the tests here!

[nchaulet]

Tested locally with an upgrade from 8.0 with few packages to that PR and looks like it working fine 🚀

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 953519a
• Storybooks Preview
• Webpack Bundle Analyzer

Metrics [docs]

✅ unchanged

History

• 💚 Build #39767 succeeded b98823f
• 💔 Build #39645 failed b453be0

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @hop-dev

[kibanamachine]

💚 All backports created successfully

Status	Branch	Result
✅	8.2

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

[joshdover]

@hop-dev If this PR doesn't make the 8.2.0 release, will this cleanup logic work if a user upgrades to 8.2.0 first where the templates aren't cleaned up and then later upgrade to 8.2.1?

[hop-dev]

will this cleanup logic work if a user upgrades to 8.2.0 first where the templates aren't cleaned up and then later upgrade to 8.2.1?

Yes, we look for legacy component templates on every package install regardless of stack version. (performance wise, in the case where there aren't any legacy templates, this will just perform a get component templates call and then exit early, so felt acceptable)