Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partial fix for benchmarks. #337

Merged
merged 3 commits into from
Dec 8, 2024
Merged

Conversation

PlugFox
Copy link
Contributor

@PlugFox PlugFox commented Dec 8, 2024

Partial fix for benchmarks.
The thing is that in some microbenchmarks the calculation result is not used in any way, because of this the compiler simply "optimizes the benchmark" by throwing away the payload.

Most of the current benchmark implementations literally measure the air in an empty room.

I can only partially fix them, I think for further fixes you should ask Vyacheslav Egorov for advice.

https://mrale.ph/blog/2021/01/21/microbenchmarking-dart-part-1.html
https://mrale.ph/blog/2024/11/27/microbenchmarks-are-experiments.html

$ dart run benchmark/vector2_bench.dart
Before After
Vector2() 202.8197194309024 us. 3732.437062937063 us.
Vector2.zero() 200.17709611451943 us. 3483.7631184407796 us.
Vector2.array() 391.3436018957346 us. 3571.7739520958085 us.
Vector2.all() 388.8081180811808 us. 3741.715034965035 us.
Vector2.copy() 386.95465071401003 us. 3838.61013986014 us.
Vector2.fromFloat32List() 389.3649748159628 us. 1719.3718140929536 us.
Vector2.fromBuffer() 5980.412087912088 us. 6143.877245508982 us.
Vector2.random() 25577.240506329115 us. 29626.848484848484 us.
Vector2.setFrom() 243.5085574572127 us. 234.90559195133363 us.
Vector2.dot() 195.64618896387455 us. 389.8708860759494 us.
$ dart compile exe -o benchmark/vector2_bench.exe benchmark/vector2_bench.dart
$ ./benchmark/vector2_bench.exe
Before After
Vector2() 3862.1923076923076 us. 3528.176911544228 us.
Vector2.zero() 194.03286978508217 us. 3356.08095952024 us.
Vector2.array() 7367.800699300699 us. 7313.762237762237 us.
Vector2.all() 3860.874125874126 us. 3485.455772113943 us.
Vector2.copy() 3835.0874125874125 us. 3624.082 us.
Vector2.fromFloat32List() 386.48084365325076 us. 1638.9460269865067 us.
Vector2.fromBuffer() 6011.754491017964 us. 7060.604895104895 us.
Vector2.random() 11294.576923076924 us. 11604.103448275862 us.
Vector2.setFrom() 519.39625 us. 349.6024287222809 us.
Vector2.dot() 196.22144129875272 us. 386.6904991327809 us.

Copy link
Collaborator

@spydon spydon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution, lgtm!

@spydon
Copy link
Collaborator

spydon commented Dec 8, 2024

@PlugFox just two analyzer infos that needs to be fixed

@coveralls
Copy link

Coverage Status

coverage: 26.372%. remained the same
when pulling 1b1108c on PlugFox:feature/fix_benchmarks
into 433fb6c on google:master.

@spydon spydon merged commit bd4b574 into google:master Dec 8, 2024
6 checks passed
copybara-service bot pushed a commit to dart-lang/sdk that referenced this pull request Dec 9, 2024
Revisions updated by `dart tools/rev_sdk_deps.dart`.

core (https://github.com/dart-lang/core/compare/abcf992..7f9f597):
  7f9f597e  2024-12-03  Devon Carew  update the pubspec version of package:collection (dart-lang/core#730)

ecosystem (https://github.com/dart-lang/ecosystem/compare/248b180..b4b2a43):
  b4b2a43  2024-12-06  Moritz  Unsubmit donotsubmit (dart-lang/ecosystem#324)
  8749a2b  2024-12-06  Moritz  Updates to `health.yaml` (dart-lang/ecosystem#310)
  10fb2c0  2024-12-05  Moritz  Ecosystem testing (dart-lang/ecosystem#320)

test (https://github.com/dart-lang/test/compare/2096773..dc0f8ea):
  dc0f8ea4  2024-12-04  Nate Bosch  Fix hang after multiple precompiled browser tests (dart-lang/test#2422)

tools (https://github.com/dart-lang/tools/compare/1c5b8b9..223daf5):
  223daf53  2024-12-05  Ben Konyi  Update `package:vm_service` constraints to >=12.0.0 <16.0.0 (dart-lang/tools#1205)

vector_math (https://github.com/google/vector_math.dart/compare/433fb6c..bd4b574):
  bd4b574  2024-12-09  Plague Fox  Partial fix for benchmarks (google/vector_math.dart#337)

Change-Id: Ie5ce1e95647ac19af1dcfd1d5ee495659052ddc9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/399701
Reviewed-by: Nate Bosch <nbosch@google.com>
Commit-Queue: Devon Carew <devoncarew@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants