-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add benchmark for libbeat diskqueue implementation #31594
Conversation
Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane) |
2d49f09
to
7e9643f
Compare
Content: beat.Event{ | ||
Timestamp: eventTime, | ||
Fields: mapstr.M{ | ||
"message": msgs[rand.Intn(len(msgs))], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does using a random event make the benchmark results less reproducible? Is there a notable improvement vs just using a fixed sequence, or maybe the compromise of using the same seed in the random number generator for every run?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A static event vs a random event doesn't make much difference with the results. I get about a 200msec variance on -benchtime 10x
runs either way.
I'll add the seed, that way order of benchmarks won't matter.
Can you summarize the results you get from this? Also, does it run on every commit? It would be nice to get the output as report attached to each CI run, but that is maybe out of scope for this change. |
} | ||
} | ||
|
||
//setup creates the disk queue, you must remove the directory it makes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a reason we can't remove it after setup? If not, we might at least want to print the dir
path so someone can manually clean it up without having to hunt around.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
updated comments
e28b105
to
985bb52
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My only question is similar to @cmacknz, how does this interact with CI? (Does it take long to run? Are the results surfaced anywhere when not running it manually?) If it doesn't have any expected "failure" outcome then should it be disabled on a default test run with a +build
directive or something?
Also FYI, depending on which goes in first this may conflict with my pending PR that removes queue.Consumer
from the queue API #31502 -- it should be an easy merge though, the only difference is that Get
is now a method on the queue itself so there's no need to create a consumer explicitly.
For results we get output like:
which is the average of 10 runs, that produced & consumed 1 million events each. or 26k events/sec. go benchmark tests do NOT run during CI, and I don't think we want them to. For this to work it would be a fair amount of I/O and I'm pretty confident that CI schedulers would limit the I/O, making any kind of comparison useless. |
Thanks, should be an easy fix. |
1743618
to
4a6cc54
Compare
* Add benchmark for libbeat diskqueue implementation
What does this PR do?
Adds a benchmark test to libbeat disqueue implementation. This benchmark produces, consumes & acks a million events to the queue.
Why is it important?
This benchmark will allow us to see how changes to queue implementation affect performance.
Checklist
- [ ] I have made corresponding changes to the documentation- [ ] I have made corresponding change to the default configuration files- [ ] I have added an entry inCHANGELOG.next.asciidoc
orCHANGELOG-developer.next.asciidoc
.How to test this PR locally
go test -bench=. -benchtime 10x -timeout 60m
Related issues