feat: add new opt-in metric of reloads by namespace #690
Conversation
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
|
@MuneebAijaz Image is available for testing. |
|
@alexanderldavis Image is available for testing. |
|
@alexanderldavis Yikes! You better fix it before anyone else finds out! Build has Failed! |
|
@alexanderldavis Yikes! You better fix it before anyone else finds out! Build has Failed! |
|
@alexanderldavis Image is available for testing. |
|
@alexanderldavis Images are available for testing. |
|
@alexanderldavis hi, can u pls take care of the unresolved comment, then we can merge this |
|
@MuneebAijaz resolved! Thank you |
Reloader currently keeps track of the number of successful and failed reloads. This is useful for general debugging, but in some cases it would also be useful to monitor reloads by namespace.
To implement this, I created a new metric (
reload_executed_total_by_namespace) withsuccessandnamespaceas attributes and disabled it by default, since the cardinality of the metric might get high if it is deployed in a cluster where Reloader manages resources in many namespaces. To keep the code changes minimal, I created a new Prometheus counter and have it increment per-namespace at the same time as the global counter, but I only register the counter if the opt-in flag is set.I've tested these changes on a cluster I manage, and verified that by default it does not expose the new metric. When the environment variable is set, it looks like this:
I've also updated the Helm chart, docs, and README to reflect this new feature.