Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add prometheus serializer and use it in prometheus output #6703

Merged
merged 9 commits into from
Nov 26, 2019

Conversation

danielnelson
Copy link
Contributor

Add a prometheus serializer, with support for round tripping metric_version=2 of the prometheus input.

related #4414

Use this serializer in the Prometheus output when metric_version=2. Formally deprecate metric_version = 1 in both the input and the output plugin.

closes #4415

Add use_batch_format support to the file output, similar to support in the amqp and kafka outputs. This is a move away from the Serialize function towards the SerializeBatch function in the Serializer interface, which allows for more efficient writing and writing multiple metrics to a destination. In regards to the prometheus serializer, the batch format is required in order to group metric families and to write histogram and summary types correctly.

There is a known bug introduced in this pull request, since histogram or summary metrics are split over multiple telegraf.Metric's, it is possible that a batch of metrics will contain only part of a histogram/summary . I documented this in the serializer docs. This issue makes me wonder if we need a way to "group" metrics for the purposes of batching?

Required for all PRs:

  • Signed CLA.
  • Associated README.md updated.
  • Has appropriate unit tests.

@danielnelson danielnelson added feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin area/prometheus new plugin labels Nov 22, 2019
@danielnelson danielnelson added this to the 1.13.0 milestone Nov 22, 2019
@danielnelson danielnelson requested a review from reimda November 22, 2019 01:37
Copy link
Contributor

@reimda reimda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a full telegraf->prometheus->telegraf test? Maybe I missed it.

@danielnelson
Copy link
Contributor Author

I'll do a roundtrip test as a follow-up pr since it will involve some changes to testutil which will then require quite a number of unittest changes.

@danielnelson danielnelson merged commit 80c5edd into master Nov 26, 2019
@danielnelson danielnelson deleted the prometheus-output-mv2 branch November 26, 2019 23:46
@danielnelson danielnelson mentioned this pull request Nov 26, 2019
3 tasks
athoune pushed a commit to bearstech/telegraf that referenced this pull request Apr 17, 2020
idohalevi pushed a commit to idohalevi/telegraf that referenced this pull request Sep 29, 2020
arstercz pushed a commit to arstercz/telegraf that referenced this pull request Mar 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/prometheus feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin new plugin
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve prometheus metric mapping to internal metrics / line protocol
2 participants