Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove old benchmarks, add new benchmark for #14394 #14626

Merged
merged 2 commits into from
Jun 2, 2024

Conversation

vkarpov15
Copy link
Collaborator

Summary

Currently the benchmarks dir is filled with old benchmarks that haven't been maintained since 2013. Given that we're looking to focus more on performance, we should remove the stale benchmarks and instead start building out a new benchmark suite based on benchmarks we've found useful, like #14394.

Here are some priorities for new benchmarks:

  1. Minimal overhead: just a plain Node.js script. No frameworks like benchmark, because benchmark adds (admittedly minimal) overhead that adds extra noise to output from tools like 0x, which I now rely on heavily for perf debugging.
  2. Includes setup code, but also allows skipping expensive setup using MONGOOSE_BENCHMARK_SKIP_SETUP environment variable. Also for flamegraphs, as well as faster iteration.
  3. Outputs only a JSON object that contains timing info, which we can then later store in MongoDB. We used to store benchmark results for our TypeScript benchmark, we disabled that temporarily but I would like to get that going again. It would be wonderful to have a website like perf.mongoosejs.io or something similar that shows Mongoose's performance on benchmarks over time.

Examples

Copy link
Collaborator

@hasezoey hasezoey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, though why is this PR targeting vkarpov15/gh-14394-2 (is it because it is build on it)?

@vkarpov15
Copy link
Collaborator Author

Yep exactly, we will merge this PR into #14623, and then merge #14263.

@vkarpov15 vkarpov15 merged commit 2d1a4e4 into vkarpov15/gh-14394-2 Jun 2, 2024
51 checks passed
@vkarpov15 vkarpov15 deleted the vkarpov15/benchmarks-v2 branch June 2, 2024 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants