Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a benchmark for the whole targets pipeline #3415

Merged

Conversation

swiatekm
Copy link
Contributor

@swiatekm swiatekm commented Nov 1, 2024

Add a benchmark for the whole scrape target processing pipeline in the target allocator. I'd like to make some changes to this pipeline in subsequent PRs that will improve its performance, and wrote this benchmark to measure the improvement. I did my best to use realistic data, and the labels in the benchmark are taken directly from a real production cluster.

Current output is as follows on my machine:

goos: linux
goarch: amd64
pkg: github.com/open-telemetry/opentelemetry-operator/cmd/otel-allocator
cpu: AMD Ryzen 9 7950X3D 16-Core Processor          
BenchmarkProcessTargets/per-node-32            	      14	  79449090 ns/op	47902082 B/op	  132380 allocs/op
BenchmarkProcessTargets/per-node-32            	      14	  80049443 ns/op	47898982 B/op	  132376 allocs/op
BenchmarkProcessTargets/per-node-32            	      14	  79190720 ns/op	47895784 B/op	  132368 allocs/op
BenchmarkProcessTargets/per-node-32            	      14	  79240592 ns/op	47914164 B/op	  132425 allocs/op
BenchmarkProcessTargets/per-node-32            	      14	  79701044 ns/op	47904677 B/op	  132402 allocs/op
BenchmarkProcessTargets/per-node-32            	      14	  79306193 ns/op	47896969 B/op	  132374 allocs/op
BenchmarkProcessTargets/per-node-32            	      14	  78996677 ns/op	47907460 B/op	  132408 allocs/op
BenchmarkProcessTargets/per-node-32            	      14	  79620516 ns/op	47893134 B/op	  132359 allocs/op
BenchmarkProcessTargets/per-node-32            	      14	  78569563 ns/op	47890921 B/op	  132352 allocs/op
BenchmarkProcessTargets/per-node-32            	      13	  79823405 ns/op	47904442 B/op	  132389 allocs/op
BenchmarkProcessTargets/least-weighted-32      	      13	  80194468 ns/op	47904215 B/op	  132394 allocs/op
BenchmarkProcessTargets/least-weighted-32      	      14	  79167548 ns/op	47910850 B/op	  132423 allocs/op
BenchmarkProcessTargets/least-weighted-32      	      13	  79440631 ns/op	47909941 B/op	  132413 allocs/op
BenchmarkProcessTargets/least-weighted-32      	      13	  79345335 ns/op	47909766 B/op	  132415 allocs/op
BenchmarkProcessTargets/least-weighted-32      	      13	  80256056 ns/op	47885332 B/op	  132332 allocs/op
BenchmarkProcessTargets/least-weighted-32      	      14	  80227125 ns/op	47905310 B/op	  132401 allocs/op
BenchmarkProcessTargets/least-weighted-32      	      13	  79355045 ns/op	47916236 B/op	  132438 allocs/op
BenchmarkProcessTargets/least-weighted-32      	      14	  78973434 ns/op	47903948 B/op	  132398 allocs/op
BenchmarkProcessTargets/least-weighted-32      	      13	  78269978 ns/op	47896699 B/op	  132368 allocs/op
BenchmarkProcessTargets/least-weighted-32      	      14	  79113204 ns/op	47905487 B/op	  132401 allocs/op
BenchmarkProcessTargets/consistent-hashing-32  	    a  14	  79156249 ns/op	47898197 B/op	  132375 allocs/op
BenchmarkProcessTargets/consistent-hashing-32  	      14	  80757290 ns/op	47908586 B/op	  132414 allocs/op
BenchmarkProcessTargets/consistent-hashing-32  	      13	  79251686 ns/op	47916901 B/op	  132441 allocs/op
BenchmarkProcessTargets/consistent-hashing-32  	      13	  79404973 ns/op	47915320 B/op	  132416 allocs/op
BenchmarkProcessTargets/consistent-hashing-32  	      14	  78935858 ns/op	47900689 B/op	  132387 allocs/op
BenchmarkProcessTargets/consistent-hashing-32  	      14	  80309503 ns/op	47885685 B/op	  132334 allocs/op
BenchmarkProcessTargets/consistent-hashing-32  	      13	  79082189 ns/op	47895481 B/op	  132366 allocs/op
BenchmarkProcessTargets/consistent-hashing-32  	      13	  79127577 ns/op	47904273 B/op	  132398 allocs/op
BenchmarkProcessTargets/consistent-hashing-32  	      14	  79708004 ns/op	47896783 B/op	  132369 allocs/op
BenchmarkProcessTargets/consistent-hashing-32  	      13	  78994510 ns/op	47903958 B/op	  132398 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/least-weighted-32         	      14	  79203552 ns/op	48205795 B/op	  132765 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/least-weighted-32         	      13	  79884622 ns/op	48222678 B/op	  132821 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/least-weighted-32         	      14	  78830938 ns/op	48208493 B/op	  132775 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/least-weighted-32         	      13	  79417045 ns/op	48208318 B/op	  132771 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/least-weighted-32         	      14	  79665405 ns/op	48213104 B/op	  132795 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/least-weighted-32         	      13	  79803822 ns/op	48215220 B/op	  132799 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/least-weighted-32         	      13	  80373446 ns/op	48214001 B/op	  132797 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/least-weighted-32         	      14	  79659748 ns/op	48224180 B/op	  132831 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/least-weighted-32         	      14	  79605281 ns/op	48203514 B/op	  132762 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/least-weighted-32         	      14	  79926999 ns/op	48211490 B/op	  132785 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/consistent-hashing-32     	      13	  77815588 ns/op	48211020 B/op	  132787 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/consistent-hashing-32     	      13	  79461895 ns/op	48214384 B/op	  132795 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/consistent-hashing-32     	      14	  79784373 ns/op	48209019 B/op	  132780 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/consistent-hashing-32     	      13	  80092055 ns/op	48209649 B/op	  132778 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/consistent-hashing-32     	      13	  79281861 ns/op	48199032 B/op	  132746 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/consistent-hashing-32     	      13	  79839818 ns/op	48216235 B/op	  132806 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/consistent-hashing-32     	      13	  79663551 ns/op	48209070 B/op	  132778 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/consistent-hashing-32     	      15	  79139760 ns/op	48192452 B/op	  132724 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/consistent-hashing-32     	      13	  79057136 ns/op	48215898 B/op	  132804 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/consistent-hashing-32     	      14	  79476125 ns/op	48210661 B/op	  132785 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/per-node-32               	      14	  79956889 ns/op	48200762 B/op	  132753 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/per-node-32               	      14	  79073909 ns/op	48231310 B/op	  132813 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/per-node-32               	      13	  79043000 ns/op	48208338 B/op	  132776 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/per-node-32               	      14	  79360126 ns/op	48207381 B/op	  132775 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/per-node-32               	      14	  79905481 ns/op	48202028 B/op	  132754 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/per-node-32               	      14	  79699799 ns/op	48199753 B/op	  132748 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/per-node-32               	      14	  79732757 ns/op	48210348 B/op	  132786 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/per-node-32               	      14	  79110739 ns/op	48201767 B/op	  132752 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/per-node-32               	      14	  78742812 ns/op	48209524 B/op	  132782 allocs/op
BenchmarkProcessTargetsWithRelabelConfig/per-node-32               	      14	  79589282 ns/op	48201804 B/op	  132757 allocs/op
PASS
ok  	github.com/open-telemetry/opentelemetry-operator/cmd/otel-allocator	70.322s

@swiatekm swiatekm added the Skip Changelog PRs that do not require a CHANGELOG.md entry label Nov 1, 2024
@swiatekm swiatekm requested a review from a team as a code owner November 1, 2024 17:59
@swiatekm swiatekm force-pushed the test/target-allocation-benchmark branch from 1fe348b to ac054ff Compare November 2, 2024 16:56
@swiatekm swiatekm merged commit 11c1fa9 into open-telemetry:main Nov 2, 2024
36 checks passed
@swiatekm swiatekm deleted the test/target-allocation-benchmark branch November 2, 2024 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Skip Changelog PRs that do not require a CHANGELOG.md entry
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants