Suggestion for the next time

[balayanv]

Reading some of the best submissions i think a lot of clever tricks people do never come to play (or even harm the numbers) because C2 will not optimise it the way they intended, imo for future submissions, top N competing solutions that get very close (instead of sharing 1st place among 3 people) should run their solutions with JMH and get more realistic and consistent numbers. Granted JMH will keep those hetzner instances sweating for a long time but it's only run for top say 5 submissions where margin of error makes it impossible to evaluate numbers with just 5 runs. Current eval method could simply act as filter and JMH invoked only on the best ones.

Love the challenge though, i've found lots of interesting crazy tricks i didn't know about.

[gunnarmorling]

Hey, thx for opening this one. Agreed that a JMH benchmark of the top 10 or so would be great, but I'm not sure whether I have the capacity to do so right now. If someone provided the actual benchmark harness/suite so that I can simply run it on the eval machine (or elsewhere), that would be amazing.

[balayanv]

If no one does before next weekend, i'll contribute one, should be trivial. Though i have not configured it for maven before.

[gunnarmorling]

That would be awesome, thanks! The challenge closes on Jan 31, i.e. it would be right on time then to get those JMH benchmarks for the top contenders.