This repo is not required as latest code for it will be maintained in https://github.com/116davinder/kafka-cluster-ansible/blob/master/roles/jmxMonitor/tasks/kafka-consumer-group-metric.yml

apache-kafka-consumer-group-metric-collector

It will collect metric about given consumer groups and reports latest_offset, committed_offset, and lag.

Requirements

• python3
• kafka-python

How to Run it

python3 kafka-consumer-group-metric.py sample-input.txt kafka-log-dir/ kafka-cluster-name

Sample Output

{
	"topic": "topic1",
	"group_id": "group_id1",
	"env": "dev-env-kafka",
	"@timestamp": "2020-05-18 15:36:01.191032",
	"partition": {
		"0": {
			"committed_offset": 13626733,
			"latest_offset": 13626733,
			"lag": 0
		},
		"1": {
			"committed_offset": 13623397,
			"latest_offset": 13623397,
			"lag": 0
		},
		"2": {
			"committed_offset": 13615804,
			"latest_offset": 13615804,
			"lag": 0
		}
	}
}
....

Sample Dashboard