Metricbeat logstash module: accept override cluster UUID from Logstash

[ycombinator]

What does this PR do?

This PR teaches the logstash Metricbeat module to accept an optional override cluster_uuid field from Logstash APIs called by the module.

If the Logstash pipeline being monitored by this module has an Elasticsearch plugin vertex in it, the cluster UUID associated with that vertex will be used. If no such vertex is found, the override cluster UUID will be used.

Why is it important?

See #15772.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works

How to test this PR locally

1. Configure Logstash to use an override cluster_uuid for monitoring. To do this, add this line to your config/logstash.yml:

   monitoring.cluster_uuid: foobar
   

2. Start a simple pipeline in Logstash.

   ./bin/logstash -e 'input { stdin {} }'
   

3. Enable the logstash-xpack module in Metricbeat. This will enable the logstash module with the required configuration for Stack Monitoring.

   ./metricbeat modules enable logstash-xpack
   

4. For easy inspection of generated events, disable the elasticsearch output and enable the console output in Metricbeat. To do this, edit your metricbeat.yml file and set output.elasticsearch.enabled: false and output.console.enabled: true.

5. Run Metricbeat and check that the generated Logstash Stack Monitoring events of type:logstash_state and type:logstash_stats both have a top-level field of cluster_uuid with the same value as set in the Logstash configuration file.

   ./metricbeat | jq -c 'select(.service.type == "logstash") | .type,.cluster_uuid'
   

Related issues

• Closes Metricbeat logstash module should honor monitoring.cluster_uuid field in Logstash API responses #15772.
• Related to Conditionally add monitoring structure to top-level of output in API logstash#11106.

[elasticmachine]

Pinging @elastic/stack-monitoring (Stack monitoring)

[urso]

Logstash pipelines are more or less independent entities (unless one forwards between pipelines). Don't we want pipeline 1 be reported with cluster 1, and pipeline 2 with cluster 2? This changes looks like all pipelines will now be reported with cluster 1 only.
In case a pipeline has no cluster UUID, we could use the pre-configured uuid as default. WDYT?

[ycombinator]

@urso Yeah, we can do that. We can invert the logic to let the pipeline's cluster UUID (if set) to override the global one. I'll make the change. Thanks!

[ycombinator]

@urso I've made the change you suggested. Thanks!

[ycombinator]

jenkins, test this

[sayden]

LGTM, tested manually and getting the following output:

➜  metricbeat git:(mb-ls-xp-cluster-uuid) ✗ ./metricbeat | jq -c 'select(.service.type == "logstash") | .type,.cluster_uuid'
"logstash_state"
"foobar"
"logstash_stats"
"foobar"
"logstash_state"
"foobar"
"logstash_state"
"foobar"
"logstash_stats"
"foobar"
"logstash_state"
"foobar"
"logstash_state"
"foobar"
"logstash_state"
"foobar"