Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In Metricbeat Kafka modules based on Jolokia, collect OneMinuteRate instead of MeanRate #17295

Closed
danielyahn opened this issue Mar 27, 2020 · 6 comments · Fixed by #22733
Closed
Labels
candidate Candidate to be added to the current iteration enhancement Metricbeat Metricbeat module Team:Integrations Label for the Integrations team Team:Services (Deprecated) Label for the former Integrations-Services team

Comments

@danielyahn
Copy link

Describe the enhancement:

  • For Metricbeat Kafka Jolokia module, we should be able to use different mbean attributes (other than the default MeanRate)
  • I think OneMinuteRate makes much more sense than MeanRate as a default value
  • Maybe make it configurable?

Describe a specific use case for the enhancement or feature:
By default, it seems like Metricbeat Kafka Jolokia module takes MeanRate attribute of metrics. A lot of times, I would prefer using OneMinuteRate to using the MeanRate. For example, name=BytesOutPerSec,type=BrokerTopicMetrics. MeanRate would have mean value of the throughput from broker start time. I want Metricbeat to ingest more granular value, and do necessary aggregation using lastic.

@andresrc andresrc added [zube]: Inbox Team:Integrations Label for the Integrations team labels Mar 29, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@andresrc
Copy link
Contributor

@jsoriano can you please comment on this?

@jsoriano
Copy link
Member

jsoriano commented Apr 1, 2020

Hey @danielyahn, thanks for your feedback!

It makes sense, it also looks to me that OneMinuteRate would make more sense for most of the metrics, as we periodically collect data and we are more interested in the most recent one.
If we see that some metric makes sense as MeanRate, we can add it, we can decide metric by metric.
I wouldn't make it configurable in the provided module, so we have control on the metrics collected for documentation, dashboards and so on.

@ChrsMark do you see any problem with collecting OneMinuteRate instead of MeanRate?

If after changing these metrics, some user still finds that some metric is not being collected as they would like, they can still use the Jolokia module directly, and the Kafka metricset manifests can be used as help to create the mappings.

@ChrsMark
Copy link
Member

ChrsMark commented Apr 1, 2020

@ChrsMark do you see any problem with collecting OneMinuteRate instead of MeanRate?

No, I don't see any problem since it makes more sense.

@jsoriano jsoriano added [zube]: Ready Team:Services (Deprecated) Label for the former Integrations-Services team and removed [zube]: Investigate labels Apr 1, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-services (Team:Services)

@jsoriano jsoriano changed the title Metricbeat Kafka Jolokia module needs to support various mbean attributes In Metricbeat Kafka modules based on Jolokia, collect OneMinuteRate instead of MeanRate Apr 1, 2020
@andresrc andresrc added the candidate Candidate to be added to the current iteration label Apr 13, 2020
@gquintana
Copy link
Contributor

MeanRate doesn't mean anything (it's too smoothed as time passes) and doesn't reflect Kafka's current status.
#22733 should fix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
candidate Candidate to be added to the current iteration enhancement Metricbeat Metricbeat module Team:Integrations Label for the Integrations team Team:Services (Deprecated) Label for the former Integrations-Services team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants