-
Notifications
You must be signed in to change notification settings - Fork 612
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Memory leaks in this exporter #193
Comments
I encountered the same and looks like it happens when connection failures occur either when connection failures occur (timeouts due to firewall / unresponsive broker under heavy load) I think it is an upstream issue, I raised an issue in Sarama to discuss this further: IBM/sarama#1857 |
We see memory gradually rising over the course of 6 days or so (9mb usage to 36mb for example), and then suddenly it will shoot up to say 150mb at which point it is restarted. Over the course of the whole period, I only seen one instance of |
In out case the connection timeout happens between prometheus scraper and this exporter. Exporter itself can scrape metrics from Kafka, but getting the page over HTTP takes a very long time. Server is quite busy with load avg staying pretty high up. We don't see the same issue on servers that are not loaded. So I don't think it is related to Sarama at all. |
Any relevant logs from the exporter @akamensky ? You can also try enabling |
Not sure under what conditions, but this exporter leaks memory like hell even causing OOM when not caught on time:
The text was updated successfully, but these errors were encountered: