[Metricbeat][Kibana][status] metrics is optional

[afharo]

What does this PR do?

Kibana might not return the metrics property to the /api/status request. This PR makes that property optional in the schema.

Why is it important?

In #20772, we've noticed some flaky tests. These changes will make Metricbeat more resilient.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

• [ ]

How to test this PR locally

I'd say we can make it work if in the test environment, Kibana takes longer than 1 minute to start, we've seen than the first green /api/status response might not contain the metrics data (it can take Kibana up to 5s to populate it after the startup).

Related issues

• Relates [Metricbeat][Kibana] Apply backoff when errored at getting usage stats #20772

Use cases

Screenshots

Logs

[elasticmachine]

💚 Build Succeeded

Expand to view the summary

Build stats

• Build Cause: [Pull request #20956 opened]

• Start Time: 2020-09-03T10:54:28.117+0000

• Duration: 75 min 47 sec

Test stats 🧪

Test	Results
Failed	0
Passed	3156
Skipped	689
Total	3845

[elasticmachine]

Pinging @elastic/stack-monitoring (Stack monitoring)

[elasticmachine]

Pinging @elastic/integrations-services (Team:Services)

[ycombinator]

LGTM.

[afharo]

@ycombinator I noticed the team @elastic/kibana-platform is planning to migrate this endpoint to Kibana's New Platform and metrics should exist in every response in the next release. Should we still go ahead and merge this PR?
elastic/kibana#76054

[ycombinator]

@afharo In that case let's not merge this PR.

[afharo]

I'll double confirm with them and will come back.

[ycombinator]

I'll double confirm with them and will come back.

@afharo Any news here?

[afharo]

Yes, @joshdover is working on elastic/kibana#76054 (that triggered elastic/kibana#76730). Both PRs should eventually ensure metrics are always available.
He mentioned to me he'd close this PR once he's done with the changes and can confirm the expected behaviour 🙂 Sorry I didn't share the news with you.

[afharo]

The fix in Kibana has been merged! elastic/kibana#76054

Closing this PR.