[Metricbeat] gcp: allow service metric prefix override

[endorama]

What does this PR do?

While working at #26824 I found out that some GCP Stackdriver metrics require a prefix which is not in the form <servicename>.googleapis.com.

This PR address this by adding a new field in the manifest.yml: service_metric_prefix. If this field is set, the value is used as metric prefix; if is not set, the previous behaviour is kept for backward compatibility.

Provide tests for the old and new behaviours.

Why is it important?

This change is required for #26824

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

• [ ]

How to test this PR locally

Related issues

• Required for [Metricbeat] gcp: add GKE metricset #26824

Use cases

Scenario: Set required value for GCP metric collection service prefix
	When a developer adds a new GCP metricset
	Given service_metric_prefix is filled
	Then service_metric_prefix value is used as metric prefix

Scenario: Empty value for GCP metric collection service prefix
	When a developer adds a new GCP metricset
	Given service_metric_prefix is left empty
	Then service_metric_prefix is computed using the provided service name

Screenshots

Logs

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2021-07-27T15:05:47.681+0000

• Duration: 110 min 48 sec

• Commit: 1444fa9

Test stats 🧪

Test	Results
Failed	0
Passed	9417
Skipped	2517
Total	11934

Trends 🧪

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	9417
Skipped	2517
Total	11934

[kaiyan-sheng]

One question here: when both service_metric_prefix and service are given, service will be ignored?

Also adding a new config option also requires updating the documentation here: https://github.com/elastic/beats/blob/master/x-pack/metricbeat/module/gcp/_meta/docs.asciidoc

[endorama]

One question here: when both service_metric_prefix and service are given, service will be ignored?

@kaiyan-sheng yes, as the explicit configuration takes priority.

We may want to deprecate the current logic based on service and require all manifests to be explicit about their metric prefix (:+1: from me on this). This is the smallest change to allow moving on GKE integration without introducing any breaking change or deprecation.

[endorama]

Also adding a new config option also requires updating the documentation here: master/x-pack/metricbeat/module/gcp/_meta/docs.asciidoc

@kaiyan-sheng I added it to the manifest.yml so, from my understanding, is not a "config" option (as in "metricbeat config").
In the asciidoc files there is no mention of other manifest related options so I'm not sure where to add it.

[kaiyan-sheng]

Also adding a new config option also requires updating the documentation here: master/x-pack/metricbeat/module/gcp/_meta/docs.asciidoc

@kaiyan-sheng I added it to the manifest.yml so, from my understanding, is not a "config" option (as in "metricbeat config").
In the asciidoc files there is no mention of other manifest related options so I'm not sure where to add it.

Since user has the freedom to use metrics metricset in gcp, this config should also be exposed under metrics (on top of adding into manifest.yml. So probably two things to add/adjust here:

1. In default config file
2. In metrics metricset specific documentation

[jsoriano]

Nice to see this done in a backwards compatible way 👍

Added some suggestions about the new setting, feel free to ignore them, not sure if I have all the context 🙂

[kaiyan-sheng]

Missing a changelog?

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b gcp-refactor-metric-prefix upstream/gcp-refactor-metric-prefix
git merge upstream/master
git push upstream gcp-refactor-metric-prefix