[Fleet] Implement agent integration health reporting

[jillguyonnet]

Summary

Implement agent integration health reporting in Fleet UI.

Closes #154634

Screenshots

These screenshots were taken with an error (invalid config) on the system integration (metrics).

Before

The error affecting the system integration is not visible in the UI. To find it, the user would need to inspect the agent JSON or run the elastic-agent status command.

After

The error affecting the system integration is surfaced in the UI:

For reference, the following screenshots show existing behaviour with the Elastic Defend integration (errors in the policy response):

Steps to reproduce

1. Enroll an agent in Fleet and add some integrations.
2. Introduce some failure, e.g. malformed package policy.
3. The failures should correctly be surfaced in the UI.

Checklist

Delete any items that are not applicable to this PR.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)
• Any UI touched in this PR does not create any new axe failures (run axe in browser: FF, Chrome)
• If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
• This renders correctly on smaller devices using a responsive layout. (You can test this in your browser)
• This was checked for cross-browser compatibility

[apmmachine]

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

• /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[karenzone]

The documentation issue is here: elastic/ingest-docs#209
@jillguyonnet, let's touch base on docs needs, time frames, and how to collaborate. As always, we're looking forward to working with you.

[jillguyonnet]

@karenzone @zombieFox

FYI I've pasted screenshots of the latest WIP iteration in the PR description above, showing various parts of the integration status tree. Note that the policy response part for Elastic Defend is not part of this change, it is existing behaviour that I am showing for reference. The change affects the health status reported on integration inputs.

Your input will be much appreciated on various points:

• Currently, only input component units are reported in this tree. The designs suggest outputs could be listed as well.
• UX for units that have no data; in the above example, in the last screenshot, the system integration has nothing to report for winlog since the agent is running on Linux. The current implementation shows a EuiHealth icon with subdued state (grey colour) and a placeholder message (Not available) which could probably be improved.
• TheEuiCallOut that reports the failure on the metrics part of the system integration has a title and a body. The body is retrieved from the agent response (that can be seen through the View agent JSON action). I put a provisional title (Failed) which might need to be discussed, improved or removed altogether. Note that there is no CTA button as it is unclear at the moment what the action should be.

cc @jlind23

[zombieFox]

Thanks @jillguyonnet for the demo. Continuing from the call:

• I just checked and EUI Callout does accept a name to change the icon. I suggest we use the new Error icon now found in EUI icon set.

[juliaElastic]

suggestion for a shorter way with lodash:

hasPackageErrors = agent.components ? some(getInputStatusFromAgent(agent.components, packagePolicy), unit => unit.status === 'DEGRADED' || unit.status === 'FAILED') : false;

[jillguyonnet]

Thanks for this @juliaElastic - there actually was some leftover unnecessary code there which I've removed.

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[juliaElastic]

I think the regex on package name might return invalid results in case we have a package name that includes the name of another. We could use a stricter regex like ^name/.

@elastic/elastic-agent-control-plane Could you suggest what would be the best way to identify component units that belong to a package/package policy?

[cmacknz]

The Elastic Agent itself only understands inputs, not packages. Any information that would allow tying an input back to a package is injected by Fleet.

Looks like you have meta.package and package_policy_id to work with. The input ID has to be unique in the agent policy and looks like it is templated from something like $inputType-$package-$packagePolicyID so that might be a good hint for what to use if you want a unique value (you can probably find the code that generates this ID faster than I can :) )

  - id: system/metrics-system-4f510cb9-2f4e-4b81-8a19-9969abe1c924
    name: system-1
    revision: 1
    type: system/metrics
    use_output: default
    meta:
      package:
        name: system
        version: 1.31.1
    data_stream:
      namespace: default
    package_policy_id: 4f510cb9-2f4e-4b81-8a19-9969abe1c924

[jillguyonnet]

Thanks for taking a look at this @juliaElastic - this is indeed one of the critical pieces of logic.
Interestingly, the yaml @cmacknz pasted above has a FullAgentPolicy type which we don't have access without an extra API call. While it would be easier (allowing us to simply retrieve units by input id), I'm not sure it's worth it. I have made a change where we only match the unit id against the package policy id; this means we only retrieve inputs, not outputs. Since we only surface the state of inputs, this should be sufficient for the present requirements.
Please let me know your thoughts.

[juliaElastic]

I agree to keep it simple and match on package policy id that works on inputs. We can come back to this later when we add support for outputs.

[juliaElastic]

Looks good overall, asked a question about the regex used to find the package units.

[jlind23]

@zombieFox @karenzone we didn't get any feedback from the demo Jill gave so I take it as an approval and we will then move forward with the current state of this PR.
Thanks for your help

[jillguyonnet]

@juliaElastic I've pushed some changes addressing your review 🙏

[juliaElastic]

LGTM 🚀

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: cdb1979

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
fleet	813	817	+4

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
fleet	970.1KB	973.3KB	+3.2KB

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
enterpriseSearch	13	15	+2
securitySolution	409	413	+4
total			+6

Total ESLint disabled count

id	before	after	diff
enterpriseSearch	14	16	+2
securitySolution	492	496	+4
total			+6

History

• 💛 Build #133744 was flaky 081b9a92ea44f32d7821643356b78808a54bd103
• 💛 Build #133685 was flaky 73053752306cf12f36736466040d262647f022d8
• 💔 Build #133320 failed 2f662c9723953e3cc685ebf8465c510de9688ca8
• 💛 Build #132984 was flaky 6ffa626223f59decfb2defcfe7bd75e9f1697643
• 💔 Build #132619 failed 70743f341f568f4ffec00a3dc3fe2f3a158cdaa4
• 💔 Build #132606 failed d8e75d45e7ba8b34cb7e915db0c11c72848f7bde

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream