-
Notifications
You must be signed in to change notification settings - Fork 150
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add NES emulation runtime benchmark #1730
Conversation
Great! I have wanted to include some kind of interpreter in the suite, an emulator should do nicely. I wanted to avoid the git dependency, but then I noticed that we already have a few of them in compile benchmarks, so it's probably not such a big deal. I wonder if in the benchmark loop, the framebuffer has to be written out? Couldn't we just benchmark the emulation of instructions itself? |
I added it because technically that is the output of the emulator, but practically speaking it probably doesn't make much difference so I can remove it if you want. (It's just a simple loop which goes over its internal palettized framebuffer, plugs each entry into a lookup table, and writes out the corresponding ABGR color.) There's also an audio callback that I've omitted, but I'm not sure whether the compiler is smart enough to delete the final parts of the audio processing pipeline if that's empty. Maybe we should |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried it without the framebuffer and the instruction count stays pretty much the same (~3.4 billion instructions). So I would just remove the framebuffer completely, and return nes
from the benchmarking closure instead of the buffer. The returned value is black_box
ed automatically by the benchmarking infrastructure.
Other than that one nit, looks good to me! I'll also let @nnethercote take a look.
Oh, one more thing, it would be nice if you included some short comment either to the top of the We should have a list of runtime benchmark descriptions in some README.md (same as for the compilation time benchmarks), but that's not prepared yet. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is a great choice for a benchmark.
Sorry for the delay! I simplified the benchmark and added more comments as requested. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, looks good to me!
I'll let Nick react to his remarks, but I think that this is good to go now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lovely, thank you!
This PR adds a new real-world runtime benchmark to the suite: Pinky, my NES emulator.
It uses a simple test ROM I've found here and emulates its execution for a given number of frames (which can be freely adjusted to make the benchmark shorter/longer).