Skip to content

Commit

Permalink
Collect OneMineRate metrics in Kafka module (elastic#22733)
Browse files Browse the repository at this point in the history
MeanRate is rate an average rate since Kafka startup, it doesn't
mean anything and doesn't shown Kafka current state.
OneMinuteRate is a better representation of current state.

(cherry picked from commit 08a7586)
  • Loading branch information
gquintana authored and jsoriano committed Dec 7, 2020
1 parent 38b917b commit f140585
Show file tree
Hide file tree
Showing 2 changed files with 36 additions and 35 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Fix failiures caused by custom beat names with more than 15 characters {pull}22550[22550]
- Stop generating NaN values from Cloud Foundry module to avoid errors in outputs. {pull}22634[22634]
- Update NATS dashboards to leverage connection and route metricsets {pull}22646[22646]
- Fix rate metrics in Kafka broker metricset by using last minute rate instead of mean rate. {pull}22733[22733]
- Fix `logstash` module when `xpack.enabled: true` is set from emitting redundant events. {pull}22808[22808]

*Packetbeat*
Expand Down
70 changes: 35 additions & 35 deletions metricbeat/module/kafka/broker/manifest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,81 +11,81 @@ input:
attributes:
- attr: Value
field: request.channel.queue.size
- mbean: 'kafka.server:name=FailedProduceRequestsPerSec,type=BrokerTopicMetrics'
- mbean: 'kafka.server:type=BrokerTopicMetrics,name=FailedProduceRequestsPerSec'
attributes:
- attr: MeanRate
- attr: OneMinuteRate
field: request.produce.failed_per_second
- mbean: 'kafka.server:name=FailedFetchRequestsPerSec,type=BrokerTopicMetrics'
- mbean: 'kafka.server:type=BrokerTopicMetrics,name=FailedFetchRequestsPerSec'
attributes:
- attr: MeanRate
- attr: OneMinuteRate
field: request.fetch.failed_per_second
- mbean: 'kafka.server:name=FailedProduceRequestsPerSec,type=BrokerTopicMetrics'
- mbean: 'kafka.server:type=BrokerTopicMetrics,name=FailedProduceRequestsPerSec'
attributes:
- attr: MeanRate
- attr: OneMinuteRate
field: request.produce.failed
- mbean: 'kafka.server:name=FailedFetchRequestsPerSec,type=BrokerTopicMetrics'
- mbean: 'kafka.server:type=BrokerTopicMetrics,name=FailedFetchRequestsPerSec'
attributes:
- attr: MeanRate
- attr: OneMinuteRate
field: request.fetch.failed
- mbean: 'kafka.controller:name=LeaderElectionRateAndTimeMs,type=ControllerStats'
- mbean: 'kafka.controller:type=ControllerStats,name=LeaderElectionRateAndTimeMs'
attributes:
- attr: MeanRate
- attr: OneMinuteRate
field: replication.leader_elections
- mbean: 'kafka.controller:type=ControllerStats,name=UncleanLeaderElectionsPerSec'
attributes:
- attr: MeanRate
- attr: OneMinuteRate
field: replication.unclean_leader_elections
- mbean: 'kafka.server:name=ZooKeeperDisconnectsPerSec,type=SessionExpireListener'
- mbean: 'kafka.server:type=SessionExpireListener,name=ZooKeeperDisconnectsPerSec'
attributes:
- attr: MeanRate
- attr: OneMinuteRate
field: session.zookeeper.disconnect
- mbean: 'kafka.server:name=ZooKeeperExpiresPerSec,type=SessionExpireListener'
- mbean: 'kafka.server:type=SessionExpireListener,name=ZooKeeperExpiresPerSec'
attributes:
- attr: MeanRate
- attr: OneMinuteRate
field: session.zookeeper.expire
- mbean: 'kafka.server:name=ZooKeeperReadOnlyConnectsPerSec,type=SessionExpireListener'
- mbean: 'kafka.server:type=SessionExpireListener,name=ZooKeeperReadOnlyConnectsPerSec'
attributes:
- attr: MeanRate
- attr: OneMinuteRate
field: session.zookeeper.readonly
- mbean: 'kafka.server:name=ZooKeeperSyncConnectsPerSec,type=SessionExpireListener'
- mbean: 'kafka.server:type=SessionExpireListener,name=ZooKeeperSyncConnectsPerSec'
attributes:
- attr: MeanRate
- attr: OneMinuteRate
field: session.zookeeper.sync
- mbean: 'kafka.log:type=LogFlushStats,name=LogFlushRateAndTimeMs'
attributes:
- attr: MeanRate
- attr: OneMinuteRate
field: log.flush_rate
- mbean: 'kafka.server:name=BytesRejectedPerSec,topic=*,type=BrokerTopicMetrics'
- mbean: 'kafka.server:type=BrokerTopicMetrics,name=BytesRejectedPerSec,topic=*'
attributes:
- attr: MeanRate
- attr: OneMinuteRate
field: topic.net.rejected.bytes_per_sec
- mbean: 'kafka.server:name=BytesInPerSec,topic=*,type=BrokerTopicMetrics,topic=*'
- mbean: 'kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec,topic=*'
attributes:
- attr: MeanRate
- attr: OneMinuteRate
field: topic.net.in.bytes_per_sec
- mbean: 'kafka.server:name=BytesOutPerSec,topic=*,type=BrokerTopicMetrics,topic=*'
- mbean: 'kafka.server:type=BrokerTopicMetrics,name=BytesOutPerSec,topic=*'
attributes:
- attr: MeanRate
- attr: OneMinuteRate
field: topic.net.out.bytes_per_sec
- mbean: 'kafka.server:type=BrokerTopicMetrics,topic=*,name=MessagesInPerSec,topic=*'
- mbean: 'kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec,topic=*'
attributes:
- attr: MeanRate
- attr: OneMinuteRate
field: topic.messages_in
- mbean: 'kafka.server:name=BytesRejectedPerSec,type=BrokerTopicMetrics'
- mbean: 'kafka.server:type=BrokerTopicMetrics,name=BytesRejectedPerSec'
attributes:
- attr: MeanRate
- attr: OneMinuteRate
field: net.rejected.bytes_per_sec
- mbean: 'kafka.server:name=BytesInPerSec,type=BrokerTopicMetrics'
- mbean: 'kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec'
attributes:
- attr: MeanRate
- attr: OneMinuteRate
field: net.in.bytes_per_sec
- mbean: 'kafka.server:name=BytesOutPerSec,type=BrokerTopicMetrics'
- mbean: 'kafka.server:type=BrokerTopicMetrics,name=BytesOutPerSec'
attributes:
- attr: MeanRate
- attr: OneMinuteRate
field: net.out.bytes_per_sec
- mbean: 'kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec'
attributes:
- attr: MeanRate
- attr: OneMinuteRate
field: messages_in


0 comments on commit f140585

Please sign in to comment.