Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port System.Threading.Performance.Tests and remove duplicated benchmarks #80

Merged
merged 4 commits into from
Jul 2, 2018

Conversation

adamsitnik
Copy link
Member

Fixes #74

Results reported by xunit-performance (Metric column removed for brevity):

System.Threading.Channels.Performance.Tests.dll Unit Iterations Average STDEV.S Min Max
System.Threading.Channels.Tests.BoundedChannelPerfTests.PingPong msec 4 2504.966 14.556 2486.916 2519.338
System.Threading.Channels.Tests.BoundedChannelPerfTests.PingPong bytes 4 -6874.000 34188.135 -39888.000 29808.000
System.Threading.Channels.Tests.BoundedChannelPerfTests.ReadAsyncThenWriteAsync msec 100 92.164 3.370 89.415 116.241
System.Threading.Channels.Tests.BoundedChannelPerfTests.ReadAsyncThenWriteAsync bytes 100 0.000 0.000 0.000 0.000
System.Threading.Channels.Tests.BoundedChannelPerfTests.TryWriteThenTryRead msec 100 49.657 0.656 48.887 53.348
System.Threading.Channels.Tests.BoundedChannelPerfTests.TryWriteThenTryRead bytes 100 0.000 0.000 0.000 0.000
System.Threading.Channels.Tests.BoundedChannelPerfTests.WriteAsyncThenReadAsync msec 100 77.709 6.011 72.436 103.316
System.Threading.Channels.Tests.BoundedChannelPerfTests.WriteAsyncThenReadAsync bytes 100 0.000 0.000 0.000 0.000
System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests.PingPong msec 5 2451.046 35.630 2414.985 2505.484
System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests.PingPong bytes 5 -440.000 29448.079 -40208.000 42952.000
System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests.ReadAsyncThenWriteAsync msec 100 91.562 0.901 90.826 97.535
System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests.ReadAsyncThenWriteAsync bytes 100 0.000 0.000 0.000 0.000
System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests.TryWriteThenTryRead msec 100 30.574 0.191 30.373 31.380
System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests.TryWriteThenTryRead bytes 100 0.000 0.000 0.000 0.000
System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests.WriteAsyncThenReadAsync msec 100 72.449 0.744 71.742 75.072
System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests.WriteAsyncThenReadAsync bytes 100 0.000 0.000 0.000 0.000
System.Threading.Channels.Tests.UnboundedChannelPerfTests.PingPong msec 4 2565.998 32.485 2525.836 2602.982
System.Threading.Channels.Tests.UnboundedChannelPerfTests.PingPong bytes 4 -7770.000 184523.815 -256784.000 189120.000
System.Threading.Channels.Tests.UnboundedChannelPerfTests.ReadAsyncThenWriteAsync msec 94 107.246 1.138 106.359 113.047
System.Threading.Channels.Tests.UnboundedChannelPerfTests.ReadAsyncThenWriteAsync bytes 94 0.000 0.000 0.000 0.000
System.Threading.Channels.Tests.UnboundedChannelPerfTests.TryWriteThenTryRead msec 100 44.448 2.991 42.320 64.041
System.Threading.Channels.Tests.UnboundedChannelPerfTests.TryWriteThenTryRead bytes 100 0.000 0.000 0.000 0.000
System.Threading.Channels.Tests.UnboundedChannelPerfTests.WriteAsyncThenReadAsync msec 100 72.118 4.670 66.332 84.937
System.Threading.Channels.Tests.UnboundedChannelPerfTests.WriteAsyncThenReadAsync bytes 100 0.000 0.000 0.000 0.000
Type Method Mean Error StdDev Median Min Max Allocated
Perf_EventWaitHandle Set_Reset 107.24 ms 1.7024 ms 1.4216 ms 106.86 ms 105.79 ms 110.22 ms 0 B
Perf_Interlocked Increment_int 50.35 ms 1.9354 ms 1.9875 ms 50.31 ms 48.50 ms 56.13 ms 0 B
Perf_Lock ReaderWriterLockSlimPerf 61.19 ms 3.5594 ms 3.9563 ms 59.89 ms 57.42 ms 70.88 ms 0 B
Perf_Monitor EnterExit 71.00 ms 1.7849 ms 1.9098 ms 69.78 ms 69.34 ms 76.15 ms 0 B
Perf_SpinLock EnterExit 49.80 ms 1.0724 ms 1.1919 ms 50.53 ms 48.24 ms 51.72 ms 0 B
Perf_Volatile Read_double 26.91 ms 0.8548 ms 0.9146 ms 26.57 ms 25.93 ms 29.30 ms 0 B
Perf_Interlocked Decrement_int 50.70 ms 1.5467 ms 1.6550 ms 50.44 ms 48.50 ms 55.12 ms 0 B
Perf_Monitor TryEnterExit 72.86 ms 1.9983 ms 2.2211 ms 73.42 ms 70.36 ms 77.79 ms 0 B
Perf_SpinLock TryEnterExit 50.46 ms 1.3674 ms 1.4631 ms 50.53 ms 48.23 ms 53.43 ms 0 B
Perf_Volatile Write_double 28.41 ms 1.9591 ms 2.2561 ms 27.84 ms 25.41 ms 33.30 ms 0 B
Perf_Interlocked Increment_long 50.47 ms 0.0595 ms 0.0556 ms 50.45 ms 50.39 ms 50.58 ms 0 B
Perf_Interlocked Decrement_long 53.91 ms 3.2214 ms 3.7098 ms 53.12 ms 48.61 ms 60.97 ms 0 B
Perf_Interlocked Add_int 51.35 ms 3.4135 ms 3.9310 ms 50.38 ms 48.22 ms 60.72 ms 0 B
Perf_Interlocked Add_long 50.10 ms 1.3942 ms 1.4318 ms 50.36 ms 48.26 ms 53.29 ms 0 B
Perf_Interlocked Exchange_int 50.79 ms 0.9942 ms 0.8814 ms 50.41 ms 50.39 ms 53.16 ms 0 B
Perf_Interlocked Exchange_long 50.82 ms 1.2076 ms 1.0084 ms 50.42 ms 50.38 ms 53.59 ms 0 B
Perf_Interlocked CompareExchange_int 79.03 ms 1.5078 ms 1.6759 ms 77.86 ms 77.33 ms 81.07 ms 0 B
Perf_Interlocked CompareExchange_long 79.40 ms 1.5782 ms 1.8174 ms 79.18 ms 77.47 ms 82.87 ms 0 B

Comment:

  1. xunit-performance needed 96,300s to run the benchmarks, BDN 109.13 sec
  2. the results are more or less the same, BDN ones have more narrow distribution
  3. again for some benchmarks xunit-performance returned negative numbers for allocated memory?!

@ViktorHofer no magic here, this was a simple case

@jorive I removed the duplicated benchmarks from CoreCLR perflab because CoreFX had more benchmarks for the Interlocked type. I moved missing benchmarks (CompareExchange_object_Match and CompareExchange_object_NoMatch) to CoreFX and added them CoreCLR category, so CLR devs can still run them with --categories=CoreCLR

…g ones to Perf.Interlocked, add CoreCLR category to make sure they are run for both repos
@adamsitnik adamsitnik requested review from ViktorHofer and jorive June 29, 2018 21:42
Copy link
Member

@jorive jorive left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:


namespace System.Threading.Tests
{
[BenchmarkCategory(Categories.CoreFX, Categories.CoreCLR)]
Copy link
Member

@jorive jorive Jul 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Categories.CoreCLR [](start = 42, length = 18)

Is this a CoreClr duplicated benchmark? nvm, I just saw the perflab tests :P #Closed

@adamsitnik adamsitnik merged commit a2aca5c into dotnet:master Jul 2, 2018
@adamsitnik adamsitnik deleted the threading branch October 17, 2018 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants