Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add Grafana dashboard #407

Merged
merged 11 commits into from
Aug 11, 2022
Merged

Conversation

HarHarLinks
Copy link
Contributor

@HarHarLinks HarHarLinks commented Jul 11, 2022

Fixes #147

image
image
image

There isn't too much going on in the pushes section, in particular I don't have jira and figma set up and i'm not sure if there are metrics for generic hooks.

I'm not a pro Grafana user, but I think it's a solid start.

A possible improvement would be to limit the y axis legends to integers which I don't know how to do yet. I used templating to easily select the correct prometheus globally, however to my knowledge the use of templating prevents one from using Grafana alerts (for now).

Signed-off-by: Kim Brose 2803622+HarHarLinks@users.noreply.github.com

@Half-Shot Half-Shot self-requested a review July 11, 2022 16:29
@Half-Shot
Copy link
Contributor

Wow, this is really cool! It's certainly much more expanded than the one I use at home :)

hookshot-dashboard.json Outdated Show resolved Hide resolved
@HarHarLinks
Copy link
Contributor Author

Wow, this is really cool! It's certainly much more expanded than the one I use at home :)

Mayhaps you can check the parts I noted in OP against a bigger instance that uses those features?

Copy link
Contributor

@Half-Shot Half-Shot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great!

docs/metrics.md Outdated Show resolved Hide resolved
Co-authored-by: Will Hunt <will@half-shot.uk>
@HarHarLinks
Copy link
Contributor Author

HarHarLinks commented Jul 14, 2022

This is giving me some trouble. E.g. I'm using rate(process_cpu_seconds_total[$__rate_interval]).

Ever since I restarted the bridge, resetting the counter, I only get "No data" from that point onward. Even though the rate() function is supposed to handle such resets:

Breaks in monotonicity (such as counter resets due to target restarts) are automatically adjusted for.

image

Similarly for the many panels using similar functions.

@HarHarLinks
Copy link
Contributor Author

I also don't understand what the CI issue is, so please advise if necessary.

@HarHarLinks
Copy link
Contributor Author

So the issue above appears related to how often my prometheus polls metrics vs the range the derivation functions ((i)rate, (i)delta, etc) are applied on. I'm still not sure why it would work before the first counter reset and not after.

I am able to work around this issue by telling Grafana these 2 global variables:
image
Interval is my sampling interval, 2x Interval is simply twice that, since I was unable to do this programmatically (it appears these don't support arithmetics).

@HarHarLinks
Copy link
Contributor Author

HarHarLinks commented Jul 18, 2022

Still some kind of trouble I don't yet understand.
image
I gotta investigate if half my trouble originates from the fact that I originally polled metrics at a much higher rate and then decreased that to 5m...

- add generic event pushes
- add http requests
- shuffle some panels around
@HarHarLinks HarHarLinks requested a review from a team as a code owner August 11, 2022 13:46
@HarHarLinks
Copy link
Contributor Author

HarHarLinks commented Aug 11, 2022

This actual data
image
when put through idelta results in this weird dip
image
instead of being thrown out as a reset and idk why it would do that.

Otherwise works well as far as I can see.

Copy link
Contributor

@Half-Shot Half-Shot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have now tried it on EMS and it looks great, one nit!

contrib/hookshot-dashboard.json Outdated Show resolved Hide resolved
@Half-Shot Half-Shot merged commit a793090 into matrix-org:main Aug 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

provide a (sample) grafana dashboard
2 participants