Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Windows/x64: 1597 Improvements on 5/19/2023 3:32:16 PM #18221

Closed
performanceautofiler bot opened this issue May 26, 2023 · 73 comments
Closed

[Perf] Windows/x64: 1597 Improvements on 5/19/2023 3:32:16 PM #18221

performanceautofiler bot opened this issue May 26, 2023 · 73 comments
Assignees
@EgorBo
Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-improvement runtime-coreclr untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 17.75 ns 12.90 ns 0.73 0.01 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int64>.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.904244688864273 < 16.86607769262662.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 222.4930579963429 (T) = (0 -12.92409463035986) / Math.Sqrt((0.14133723661728373 / (299)) + (0.000788933903898455 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2844835583542544 = (18.062610274382234 - 12.92409463035986) / 18.062610274382234 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Node.Tests.Perf_ParseThenWrite

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseThenWrite - Duration of single invocation 6.32 μs 4.64 μs 0.73 0.03 True
ParseThenWrite - Duration of single invocation 54.21 μs 42.18 μs 0.78 0.05 True
ParseThenWrite - Duration of single invocation 7.30 μs 5.13 μs 0.70 0.12 True
ParseThenWrite - Duration of single invocation 38.31 μs 29.96 μs 0.78 0.06 True
ParseThenWrite - Duration of single invocation 8.89 μs 5.68 μs 0.64 0.10 False
ParseThenWrite - Duration of single invocation 701.14 ns 533.66 ns 0.76 0.09 False
ParseThenWrite - Duration of single invocation 746.87 ns 554.65 ns 0.74 0.06 False
ParseThenWrite - Duration of single invocation 41.24 μs 29.66 μs 0.72 0.03 True
ParseThenWrite - Duration of single invocation 5.71 μs 4.26 μs 0.75 0.03 True
ParseThenWrite - Duration of single invocation 61.53 μs 46.25 μs 0.75 0.04 True
ParseThenWrite - Duration of single invocation 13.61 μs 10.18 μs 0.75 0.07 True
ParseThenWrite - Duration of single invocation 30.84 μs 24.87 μs 0.81 0.02 False
ParseThenWrite - Duration of single invocation 15.70 μs 10.57 μs 0.67 0.04 True
ParseThenWrite - Duration of single invocation 37.07 μs 26.71 μs 0.72 0.03 True

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Node.Tests.Perf_ParseThenWrite*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json400B)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.639916174929686 < 5.911508184086717.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 69.41660054511834 (T) = (0 -4501.565458347357) / Math.Sqrt((11131.760479322618 / (299)) + (6069.1550449482775 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2571797223298137 = (6060.100395301892 - 4501.565458347357) / 6060.100395301892 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: BroadTree)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 42.18184886198004 < 52.162314548059264.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 44.41557193889819 (T) = (0 -41951.311264526594) / Math.Sqrt((1003553.945468551 / (299)) + (873183.7150307638 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21945962180006107 = (53746.49721680456 - 41951.311264526594) / 53746.49721680456 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfStrings)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.133807837971349 < 6.991996609279628.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 91.24718288122058 (T) = (0 -5029.099029209677) / Math.Sqrt((57802.8891369074 / (299)) + (4875.148557710863 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3019412051684129 = (7204.406085053325 - 5029.099029209677) / 7204.406085053325 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: DeepTree)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 29.95914308553158 < 36.34152829994659.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 115.84813649363366 (T) = (0 -29797.16604749674) / Math.Sqrt((483099.21474681544 / (299)) + (42416.87799538316 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2135620482375628 = (37888.769203877004 - 29797.16604749674) / 37888.769203877004 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfStrings)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.67700890189761 < 8.295360371166804.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 90.70789466565274 (T) = (0 -5704.815013800543) / Math.Sqrt((107051.7728041298 / (299)) + (5391.1747663339465 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.30651750461356064 = (8226.328785157822 - 5704.815013800543) / 8226.328785157822 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: HelloWorld)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 533.6620154116633 < 679.1483454424651.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 98.00889980649804 (T) = (0 -523.5853356596598) / Math.Sqrt((449.4869398774267 / (299)) + (21.174371025235843 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24884344792257565 = (697.038898498075 - 523.5853356596598) / 697.038898498075 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: HelloWorld)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 554.6478225950053 < 709.8344592305807.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 42.15168465552551 (T) = (0 -562.7972595993093) / Math.Sqrt((365.49386742425344 / (299)) + (186.56688701894765 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22813828676522963 = (729.1426041080603 - 562.7972595993093) / 729.1426041080603 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json4KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 29.65888757489751 < 39.49511470637908.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 81.70108669839428 (T) = (0 -30284.413969925157) / Math.Sqrt((480352.8504411192 / (299)) + (213964.42449424884 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26611063927730594 = (41265.63974180348 - 30284.413969925157) / 41265.63974180348 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json400B)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.26481497269064 < 5.422901561264555.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 67.23332976651228 (T) = (0 -4142.455853355492) / Math.Sqrt((12084.262413667122 / (299)) + (6050.3056717682675 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.267414362094922 = (5654.568748032479 - 4142.455853355492) / 5654.568748032479 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: BroadTree)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 46.2503810754503 < 58.32557946146402.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 55.17870097098882 (T) = (0 -46837.874529926965) / Math.Sqrt((1398364.5892493192 / (299)) + (816033.5317934522 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23427817667451972 = (61168.26385659626 - 46837.874529926965) / 61168.26385659626 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfNumbers)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.181023923831042 < 13.164760122478764.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 44.949835914656916 (T) = (0 -10037.709576986457) / Math.Sqrt((91137.78827877126 / (299)) + (90805.80482450855 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2765906574381648 = (13875.559778423043 - 10037.709576986457) / 13875.559778423043 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: DeepTree)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 24.86885686025152 < 29.391899020329607.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/20/2023 3:57:43 PM, 5/21/2023 1:46:51 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 44.15567499745064 (T) = (0 -24504.669630937453) / Math.Sqrt((1205281.585442977 / (299)) + (67370.1546515624 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.2065615172755248 = (30884.145607349878 - 24504.669630937453) / 30884.145607349878 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfNumbers)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.565066229342753 < 14.777126333892623.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 61.027014354707106 (T) = (0 -10810.993726102319) / Math.Sqrt((85173.91556295483 / (299)) + (63418.409598312224 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2885686068459004 = (15196.115648161458 - 10810.993726102319) / 15196.115648161458 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json4KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 26.714912673851952 < 35.53891081919664.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 125.58360972441801 (T) = (0 -27022.777367542047) / Math.Sqrt((347425.1957470663 / (299)) + (63367.04959742713 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2652833123278436 = (36779.86061968976 - 27022.777367542047) / 36779.86061968976 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SystemTextJson_Reflection_ - Duration of single invocation 342.57 μs 260.92 μs 0.76 0.04 True
JsonNet_ - Duration of single invocation 734.88 μs 504.51 μs 0.69 0.09 True
SystemTextJson_SourceGen_ - Duration of single invocation 340.70 μs 255.15 μs 0.75 0.08 True
DataContractJsonSerializer_ - Duration of single invocation 3.09 ms 2.30 ms 0.74 0.10 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>.SystemTextJson_Reflection_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 260.92207422823446 < 330.36291898792933.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 34.46907228834079 (T) = (0 -266766.0609627053) / Math.Sqrt((31714605.324341383 / (299)) + (65174383.61923811 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2262194257209884 = (344756.730564438 - 266766.0609627053) / 344756.730564438 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>.JsonNet_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 504.50620782726037 < 707.9807408613972.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 80.92957265912557 (T) = (0 -510800.0850801197) / Math.Sqrt((395873359.72450274 / (299)) + (101698877.36986107 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.32394799550592535 = (755563.3023562712 - 510800.0850801197) / 755563.3023562712 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>.SystemTextJson_SourceGen_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 255.15013689253934 < 325.9795923913044.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 75.43002226367041 (T) = (0 -256118.05990896316) / Math.Sqrt((58423048.12629384 / (299)) + (15369758.022177288 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2568832876344983 = (344653.88228678616 - 256118.05990896316) / 344653.88228678616 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>.DataContractJsonSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.2957994708994707 < 2.91951533127572.
IsChangePoint: Marked as a change because one of 4/17/2023 1:30:24 AM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 54.03119311101135 (T) = (0 -2287012.9103445755) / Math.Sqrt((11723215828.622349 / (299)) + (2700272522.1950817 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27073328615261694 = (3136044.559443843 - 2287012.9103445755) / 3136044.559443843 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CtorInvoke - Duration of single invocation 6.37 μs 5.71 μs 0.90 0.01 True
Date_IsNotMatch - Duration of single invocation 393.28 ns 269.19 ns 0.68 0.13 False
MatchesWords - Duration of single invocation 2.99 μs 2.65 μs 0.88 0.02 False
Backtracking - Duration of single invocation 107.49 ns 90.55 ns 0.84 0.09 False
MatchWord - Duration of single invocation 151.54 ns 133.99 ns 0.88 0.02 False
MatchesSet - Duration of single invocation 160.66 μs 106.11 μs 0.66 0.22 False
Date_IsMatch - Duration of single invocation 69.66 ns 59.51 ns 0.85 0.07 False
ReplaceWords - Duration of single invocation 56.77 μs 43.43 μs 0.77 0.17 False
MatchesWords - Duration of single invocation 57.76 μs 43.24 μs 0.75 0.32 False
Uri_IsMatch - Duration of single invocation 81.29 ns 76.17 ns 0.94 0.08 False
Uri_IsNotMatch - Duration of single invocation 335.33 ns 261.77 ns 0.78 0.12 False
Backtracking - Duration of single invocation 128.14 ns 109.08 ns 0.85 0.07 False
MatchesBoundary - Duration of single invocation 118.39 μs 97.66 μs 0.82 0.22 False
SplitWords - Duration of single invocation 61.12 μs 43.63 μs 0.71 0.27 False
SplitWords - Duration of single invocation 2.86 μs 2.67 μs 0.94 0.02 False
Backtracking - Duration of single invocation 1.97 μs 1.10 μs 0.56 0.07 False
Email_IsNotMatch - Duration of single invocation 630.12 ns 418.04 ns 0.66 0.13 True
Ctor - Duration of single invocation 32.90 μs 26.38 μs 0.80 0.01 False
MatchesWord - Duration of single invocation 1.30 μs 1.05 μs 0.80 0.11 False
IP_IsNotMatch - Duration of single invocation 632.14 ns 510.79 ns 0.81 0.05 False
Date_IsMatch - Duration of single invocation 68.72 ns 58.35 ns 0.85 0.06 False
MatchWord - Duration of single invocation 1.73 μs 1.23 μs 0.71 0.18 False
Date_IsMatch - Duration of single invocation 173.87 ns 119.87 ns 0.69 0.11 False
Ctor - Duration of single invocation 6.28 μs 5.47 μs 0.87 0.01 True
Uri_IsMatch - Duration of single invocation 283.93 ns 215.12 ns 0.76 0.05 False
MatchesWords - Duration of single invocation 3.16 μs 2.78 μs 0.88 0.02 True
Ctor - Duration of single invocation 24.07 μs 20.47 μs 0.85 0.01 False
OneNodeBacktracking - Duration of single invocation 1.04 μs 776.28 ns 0.75 0.02 True
IP_IsMatch - Duration of single invocation 626.68 ns 499.23 ns 0.80 0.04 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.711254145173176 < 6.0446357709276715.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 40.96878804634076 (T) = (0 -5627.458630349488) / Math.Sqrt((4739.621645374509 / (299)) + (4993.895413135041 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12709701651750646 = (6446.831706197679 - 5627.458630349488) / 6446.831706197679 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 269.18985100641606 < 374.6954909942867.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 28.079322204531802 (T) = (0 -278.32375614504275) / Math.Sqrt((225.44206721945926 / (299)) + (199.0097748473713 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2879194978427685 = (390.8599593751933 - 278.32375614504275) / 390.8599593751933 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.6451178941205877 < 2.8293227248934683.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 58.77186055209188 (T) = (0 -2664.347136508407) / Math.Sqrt((1993.0582111843423 / (299)) + (224.1403946030379 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09735555765740807 = (2951.712780276748 - 2664.347136508407) / 2951.712780276748 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 90.54868123382309 < 101.7413103536039.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/20/2023 8:31:51 AM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 13.880779024924424 (T) = (0 -94.40052609116634) / Math.Sqrt((37.45265503025974 / (299)) + (20.63227642972695 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.12178486657579335 = (107.49134522778463 - 94.40052609116634) / 107.49134522778463 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: IgnoreCase, Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 133.99387958023092 < 144.05236376600575.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/3/2023 7:05:50 AM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 17.488016742693237 (T) = (0 -136.6356206449521) / Math.Sqrt((9.363204560330564 / (299)) + (10.404073877598037 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.07755228041490994 = (148.1228884238662 - 136.6356206449521) / 148.1228884238662 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesSet(Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 106.11087781266353 < 151.31043440934067.
IsChangePoint: Marked as a change because one of 4/18/2023 7:42:00 PM, 4/24/2023 8:06:00 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 76.89584630655183 (T) = (0 -106325.67106507962) / Math.Sqrt((108334949.28927831 / (299)) + (129895.6643722675 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.30617546197967255 = (153245.7635016139 - 106325.67106507962) / 153245.7635016139 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 59.50800376647835 < 66.1883211851953.
IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 3/14/2023 8:42:09 PM, 5/16/2023 12:30:38 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 54.28214322400863 (T) = (0 -59.11723217351679) / Math.Sqrt((12.980737943640099 / (299)) + (0.05722193976617255 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16721019941976975 = (70.98697910604572 - 59.11723217351679) / 70.98697910604572 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 43.4348385040374 < 54.14704354142207.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 96.01168657876288 (T) = (0 -42865.9137141632) / Math.Sqrt((6408695.337823642 / (299)) + (60768.36076783024 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2657372407495051 = (58379.52854631897 - 42865.9137141632) / 58379.52854631897 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 43.24120898146175 < 54.95127582211477.
IsChangePoint: Marked as a change because one of 4/3/2023 7:47:24 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 32.76240777139453 (T) = (0 -43604.64571437115) / Math.Sqrt((9471900.73315233 / (299)) + (2810903.622609421 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27224879070570773 = (59916.96771848036 - 43604.64571437115) / 59916.96771848036 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 76.17336601204987 < 77.29234002130602.
IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 4/17/2023 6:40:18 PM, 5/5/2023 3:18:41 PM, 5/10/2023 2:00:36 PM, 5/16/2023 12:30:38 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 36.5972323630553 (T) = (0 -76.28608957016637) / Math.Sqrt((17.883028511058615 / (299)) + (0.0656475198625797 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1088748603495917 = (85.60648350701192 - 76.28608957016637) / 85.60648350701192 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 261.76847221524184 < 312.3010047053495.
IsChangePoint: Marked as a change because one of 4/26/2023 11:48:46 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 19.09423284911391 (T) = (0 -270.3604606302068) / Math.Sqrt((180.64611357681693 / (299)) + (48.474965553801326 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1281698360290069 = (310.10679809330617 - 270.3604606302068) / 310.10679809330617 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: IgnoreCase, Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 109.08284601114808 < 121.77952034278181.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 45.03198819788925 (T) = (0 -109.52224166100869) / Math.Sqrt((28.914896708017704 / (299)) + (1.4145699406095484 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15711342698902445 = (129.93710561762924 - 109.52224166100869) / 129.93710561762924 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesBoundary(Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 97.65683681491467 < 112.66773017416743.
IsChangePoint: Marked as a change because one of 3/25/2023 10:49:06 AM, 3/28/2023 3:45:48 AM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 58.145347930566416 (T) = (0 -98880.98977099602) / Math.Sqrt((72857660.01702362 / (299)) + (511028.1790411933 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23827521335730378 = (129811.96293587112 - 98880.98977099602) / 129811.96293587112 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 43.62880652260904 < 57.08246236127288.
IsChangePoint: Marked as a change because one of 4/7/2023 3:57:10 PM, 4/30/2023 6:17:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 71.58653206195441 (T) = (0 -43129.213199307145) / Math.Sqrt((16035642.967347074 / (299)) + (101670.41435070588 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2915133556883133 = (60875.12523430601 - 43129.213199307145) / 60875.12523430601 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: IgnoreCase, Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.6747405346239566 < 2.715234413980075.
IsChangePoint: Marked as a change because one of 3/3/2023 4:59:02 PM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 10:36:00 PM, 4/9/2023 11:04:55 PM, 5/10/2023 6:18:38 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 45.400755176071016 (T) = (0 -2682.68961656012) / Math.Sqrt((5107.889787905349 / (299)) + (137.55200302135563 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0817369718330231 = (2921.482771570653 - 2682.68961656012) / 2921.482771570653 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0952287731121912 < 1.8479411331467055.
IsChangePoint: Marked as a change because one of 5/1/2023 3:42:23 PM, 5/9/2023 1:22:51 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 87.72055843076855 (T) = (0 -1092.814343443684) / Math.Sqrt((21144.061816902977 / (299)) + (409.5994274538975 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.44799521431944445 = (1979.718965835369 - 1092.814343443684) / 1979.718965835369 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 418.043564453473 < 588.355541656465.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 62.818997443109076 (T) = (0 -416.14395185575245) / Math.Sqrt((246.92068200528945 / (299)) + (129.14990716089076 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.33118376161813445 = (622.2097012216261 - 416.14395185575245) / 622.2097012216261 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options: IgnoreCase, Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 26.378361330911922 < 30.946321325088395.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 60.79348136170598 (T) = (0 -26344.262547179984) / Math.Sqrt((979411.5675084223 / (299)) + (53511.11871926699 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16555606992392488 = (31571.03982382401 - 26344.262547179984) / 31571.03982382401 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0451795964403334 < 1.2041650556410508.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/17/2023 6:40:18 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 10.32856777272213 (T) = (0 -1080.2924777755056) / Math.Sqrt((4988.396743929031 / (299)) + (2542.3108852658643 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.122259996018091 = (1230.7659134535372 - 1080.2924777755056) / 1230.7659134535372 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 510.792782859632 < 596.1381947403688.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 26.332545956297743 (T) = (0 -517.6187128595941) / Math.Sqrt((319.62674890976814 / (299)) + (323.3434733840031 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20578699540162898 = (651.737896335947 - 517.6187128595941) / 651.737896335947 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: IgnoreCase, Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 58.353127618313344 < 65.21206410363894.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/9/2023 11:04:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 37.66223094188629 (T) = (0 -58.57708177024294) / Math.Sqrt((30.0855456821803 / (299)) + (0.048276478965463325 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17196504547329697 = (70.74228140975649 - 58.57708177024294) / 70.74228140975649 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2305530021721454 < 1.6391756948407603.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 103.96234128533428 (T) = (0 -1226.1702438532734) / Math.Sqrt((4600.150828249248 / (299)) + (30.403681868720767 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26305187809843017 = (1663.8487939820632 - 1226.1702438532734) / 1663.8487939820632 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 119.8692902167343 < 155.78017866524317.
IsChangePoint: Marked as a change because one of 5/9/2023 1:09:22 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 84.9063756706076 (T) = (0 -119.91137083820696) / Math.Sqrt((43.61371616060261 / (299)) + (0.5486527208899622 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23493066802038703 = (156.73268529524887 - 119.91137083820696) / 156.73268529524887 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.467413749322861 < 5.94286048425961.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 42.670844428548556 (T) = (0 -5461.476495268742) / Math.Sqrt((4691.976585640059 / (299)) + (4498.081109083847 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12936858738418994 = (6273.006482570791 - 5461.476495268742) / 6273.006482570791 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 215.1158969969427 < 269.6971600505171.
IsChangePoint: Marked as a change because one of 3/25/2023 10:49:06 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 42.480908639201004 (T) = (0 -220.9307952245693) / Math.Sqrt((187.2097887334944 / (299)) + (37.52856057804453 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26491653678866006 = (300.551986653862 - 220.9307952245693) / 300.551986653862 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: IgnoreCase, Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.777307416003537 < 2.9843960698501846.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 51.6348020590443 (T) = (0 -2771.417075177324) / Math.Sqrt((2901.404923047323 / (299)) + (749.4241869636427 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13262405815653794 = (3195.174020260629 - 2771.417075177324) / 3195.174020260629 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.469640525923165 < 22.8158225340765.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 40.74464555614987 (T) = (0 -20327.09227050109) / Math.Sqrt((813617.4433809184 / (299)) + (24004.769445927803 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11930706787053824 = (23080.794143937797 - 20327.09227050109) / 23080.794143937797 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 776.2785681369825 < 0.9895325169265522.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 70.99707146428575 (T) = (0 -779.3354211833745) / Math.Sqrt((674.8455734525477 / (299)) + (177.74872922741426 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2666470503335685 = (1062.7016930086097 - 779.3354211833745) / 1062.7016930086097 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 499.227777993637 < 601.0609778420609.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 21.500626812849074 (T) = (0 -514.5912914782161) / Math.Sqrt((195.0603332269449 / (299)) + (517.7846254446591 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21001041475193785 = (651.3899690419728 - 514.5912914782161) / 651.3899690419728 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<ClassImplementingIXmlSerialiable>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractSerializer_BinaryXml_ - Duration of single invocation 400.83 ns 253.65 ns 0.63 0.10 False
XmlSerializer_ - Duration of single invocation 948.81 ns 806.57 ns 0.85 0.08 False
DataContractSerializer_ - Duration of single invocation 569.29 ns 395.50 ns 0.69 0.09 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;ClassImplementingIXmlSerialiable&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<ClassImplementingIXmlSerialiable>.DataContractSerializer_BinaryXml_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 253.64933652557718 < 373.8903191023158.
IsChangePoint: Marked as a change because one of 4/4/2023 8:16:00 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 64.57062541144593 (T) = (0 -258.4092899045759) / Math.Sqrt((575.0905130690887 / (299)) + (55.91165059512375 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3840113146511552 = (419.50330590607257 - 258.4092899045759) / 419.50330590607257 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_ToStream<ClassImplementingIXmlSerialiable>.XmlSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 806.5738482033184 < 907.6773465174294.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 29.082977342888455 (T) = (0 -832.21375257502) / Math.Sqrt((461.393305011115 / (299)) + (316.7715203972501 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15102059919708674 = (980.2519964418013 - 832.21375257502) / 980.2519964418013 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_ToStream<ClassImplementingIXmlSerialiable>.DataContractSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 395.4980915190502 < 541.6079035741012.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 50.74606963805476 (T) = (0 -402.21277663016275) / Math.Sqrt((190.90394103062786 / (299)) + (167.21156632733033 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3167455014275321 = (588.6719772361703 - 402.21277663016275) / 588.6719772361703 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromStream - Duration of single invocation 42.63 μs 34.84 μs 0.82 0.04 True
DeserializeFromUtf8Bytes - Duration of single invocation 43.01 μs 33.18 μs 0.77 0.06 True
DeserializeFromReader - Duration of single invocation 47.06 μs 35.01 μs 0.74 0.03 True
DeserializeFromStream - Duration of single invocation 43.07 μs 33.81 μs 0.78 0.06 True
DeserializeFromUtf8Bytes - Duration of single invocation 40.86 μs 32.53 μs 0.80 0.07 True
DeserializeFromString - Duration of single invocation 43.29 μs 32.26 μs 0.75 0.06 True
DeserializeFromString - Duration of single invocation 42.53 μs 34.43 μs 0.81 0.04 True
DeserializeFromReader - Duration of single invocation 46.40 μs 35.78 μs 0.77 0.05 True

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 34.835211214347446 < 41.35507936486679.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 51.904947253400714 (T) = (0 -34665.283206784734) / Math.Sqrt((581751.8106053787 / (299)) + (324283.28382665286 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19713224502998647 = (43176.828303534814 - 34665.283206784734) / 43176.828303534814 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 33.179515360169496 < 39.631527235721556.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 44.36492135406453 (T) = (0 -33346.08250223556) / Math.Sqrt((630013.2371228359 / (299)) + (410190.9452560081 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19620085416875258 = (41485.59086580114 - 33346.08250223556) / 41485.59086580114 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromReader(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 35.00731130073017 < 44.79041431225025.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 16.63651278967228 (T) = (0 -37378.13163708176) / Math.Sqrt((527246.0179238516 / (299)) + (3804935.3971646875 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19453497513745693 = (46405.65447700294 - 37378.13163708176) / 46405.65447700294 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 33.80515959555909 < 41.18904070899405.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 28.841905149070257 (T) = (0 -34675.091909770585) / Math.Sqrt((724215.1927428815 / (299)) + (1181062.2221969613 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20257070242416397 = (43483.59411321097 - 34675.091909770585) / 43483.59411321097 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 32.53283002804487 < 38.729151981570354.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 52.426654826091486 (T) = (0 -32412.035286106442) / Math.Sqrt((925322.7546433466 / (299)) + (339583.752605719 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2165943115064471 = (41373.24474683487 - 32412.035286106442) / 41373.24474683487 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 32.26358873906705 < 40.24309131979817.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 31.07205683440477 (T) = (0 -33630.40329596719) / Math.Sqrt((613866.9909700807 / (299)) + (888465.1180896913 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1968772391803807 = (41874.548869273734 - 33630.40329596719) / 41874.548869273734 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 34.42749408143939 < 40.20452133566493.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 31.170897394153847 (T) = (0 -33671.67637381779) / Math.Sqrt((627650.7615721698 / (299)) + (921249.1956973824 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20003696601391296 = (42091.54041285789 - 33671.67637381779) / 42091.54041285789 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromReader(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 35.78193054833681 < 44.27092025812729.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 59.49036182120755 (T) = (0 -36302.53478719417) / Math.Sqrt((752462.0856178192 / (299)) + (327751.83185670007 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2143797481610174 = (46208.75633261372 - 36302.53478719417) / 46208.75633261372 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.StringEquality

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Compare_DifferentFirstChar - Duration of single invocation 13.02 ns 9.22 ns 0.71 0.05 False
Compare_DifferentFirstChar - Duration of single invocation 13.17 ns 8.15 ns 0.62 0.03 False
Compare_Same_Upper - Duration of single invocation 15.40 ns 12.32 ns 0.80 0.04 False
Compare_Same_Upper - Duration of single invocation 1.43 μs 1.26 μs 0.88 0.01 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringEquality*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringEquality.Compare_DifferentFirstChar(Count: 1024, Options: (en-US, OrdinalIgnoreCase))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.222716353997763 < 12.369948688695652.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 5/10/2023 6:18:38 PM, 5/11/2023 5:02:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 87.87574697965769 (T) = (0 -9.229854153383151) / Math.Sqrt((0.5485530989846537 / (299)) + (0.00024024885843491322 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.29070441585347556 = (13.01270494231143 - 9.229854153383151) / 13.01270494231143 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringEquality.Compare_DifferentFirstChar(Count: 1024, Options: (en-US, Ordinal))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.148913398951379 < 12.512468891499172.
IsChangePoint: Marked as a change because one of 5/10/2023 6:18:38 PM, 5/11/2023 5:02:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 86.09784344315258 (T) = (0 -8.163213908946208) / Math.Sqrt((0.6894896234260313 / (299)) + (0.012213233760111054 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3753337380881742 = (13.068120381533394 - 8.163213908946208) / 13.068120381533394 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, Ordinal))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.321044563279365 < 14.615939448593135.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/16/2023 8:14:39 PM, 5/5/2023 3:18:41 PM, 5/16/2023 12:30:38 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 64.77965892425331 (T) = (0 -12.214415969868934) / Math.Sqrt((0.9867789396573353 / (299)) + (0.010048242482088078 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2528843521468212 = (16.348762075813568 - 12.214415969868934) / 16.348762075813568 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, OrdinalIgnoreCase))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2569342439889024 < 1.3614716016951351.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/12/2023 8:39:35 PM, 5/10/2023 6:18:38 PM, 5/11/2023 5:02:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 28.539005562729844 (T) = (0 -1240.047143570024) / Math.Sqrt((1574.2189904278287 / (299)) + (699.6609044304728 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15031351455263622 = (1459.4172848555236 - 1240.047143570024) / 1459.4172848555236 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<StructRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromString - Duration of single invocation 855.39 ns 568.08 ns 0.66 0.08 True
DeserializeFromStream - Duration of single invocation 1.18 μs 763.78 ns 0.65 0.06 True
DeserializeFromReader - Duration of single invocation 1.16 μs 842.08 ns 0.72 0.03 True
DeserializeFromStream - Duration of single invocation 1.16 μs 745.99 ns 0.64 0.01 True
DeserializeFromUtf8Bytes - Duration of single invocation 738.96 ns 512.71 ns 0.69 0.08 True
DeserializeFromUtf8Bytes - Duration of single invocation 740.81 ns 509.05 ns 0.69 0.05 True
DeserializeFromReader - Duration of single invocation 1.16 μs 806.21 ns 0.70 0.07 True
DeserializeFromString - Duration of single invocation 809.93 ns 558.64 ns 0.69 0.09 True

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;StructRecord&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 568.08438315021 < 792.0822221487332.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 89.09478880400493 (T) = (0 -576.5579484701657) / Math.Sqrt((522.3926014128904 / (299)) + (82.2516048798782 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3051188582217693 = (829.7216801635012 - 576.5579484701657) / 829.7216801635012 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 763.7762395458938 < 1.1327285685872235.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 168.78951168228218 (T) = (0 -757.1389863240407) / Math.Sqrt((427.2542472491573 / (299)) + (65.28323282125714 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3615143059535182 = (1185.8354750684844 - 757.1389863240407) / 1185.8354750684844 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromReader(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 842.082252875094 < 1.1220922782320337.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 95.52649171412814 (T) = (0 -821.5113878152562) / Math.Sqrt((647.9154122896942 / (299)) + (142.19866245157633 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.29624700912962176 = (1167.3291601918995 - 821.5113878152562) / 1167.3291601918995 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 745.9875336765132 < 1.1124742466176767.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 178.34540348702626 (T) = (0 -752.4703709732318) / Math.Sqrt((366.9741290218248 / (299)) + (57.00091351374173 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.35958086433009806 = (1174.965470365154 - 752.4703709732318) / 1174.965470365154 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 512.7074833645182 < 702.0373994984561.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 70.60707417049896 (T) = (0 -512.3426939810681) / Math.Sqrt((297.2752327973869 / (299)) + (110.86166023234864 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.29837285766211435 = (730.2207441318412 - 512.3426939810681) / 730.2207441318412 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 509.0463017126516 < 712.2584810167842.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 57.12548592975586 (T) = (0 -519.8202731364789) / Math.Sqrt((529.0964084612042 / (299)) + (176.70326381308325 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3010546682081227 = (743.7209313692994 - 519.8202731364789) / 743.7209313692994 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromReader(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 806.209557317845 < 1.106733857516145.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 86.10828967442147 (T) = (0 -808.5058365426784) / Math.Sqrt((539.4032786432598 / (299)) + (202.01299040133347 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.30725717165309163 = (1167.108201570293 - 808.5058365426784) / 1167.108201570293 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 558.6375300446526 < 786.196819536374.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 71.40238721247972 (T) = (0 -571.1208845095887) / Math.Sqrt((471.65617761523333 / (299)) + (131.26603338834678 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2993171054284673 = (815.0918039162764 - 571.1208845095887) / 815.0918039162764 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ElementAt - Duration of single invocation 153.06 ns 118.32 ns 0.77 0.19 False
Aggregate - Duration of single invocation 633.04 ns 259.86 ns 0.41 0.34 False
OrderBy - Duration of single invocation 2.91 μs 1.61 μs 0.55 0.01 True
SingleWithPredicate_FirstElementMatches - Duration of single invocation 551.36 ns 259.35 ns 0.47 0.34 False
SelectToList - Duration of single invocation 329.93 ns 111.30 ns 0.34 0.09 True
AnyWithPredicate_LastElementMatches - Duration of single invocation 575.72 ns 255.22 ns 0.44 0.27 False
OrderByThenBy - Duration of single invocation 4.27 μs 1.92 μs 0.45 0.02 True
Contains_ElementNotFound - Duration of single invocation 412.04 ns 258.89 ns 0.63 0.04 True
LastWithPredicate_FirstElementMatches - Duration of single invocation 614.30 ns 252.40 ns 0.41 0.11 True
TakeLastHalf - Duration of single invocation 1.21 μs 935.56 ns 0.77 0.01 True
AnyWithPredicate_LastElementMatches - Duration of single invocation 2.38 μs 1.40 μs 0.59 0.01 True
WhereSingle_LastElementMatches - Duration of single invocation 346.45 ns 255.09 ns 0.74 0.05 True
ToArray - Duration of single invocation 708.03 ns 427.88 ns 0.60 0.06 True
Reverse - Duration of single invocation 1.23 μs 569.22 ns 0.46 0.13 True
ToArray - Duration of single invocation 38.47 ns 31.65 ns 0.82 0.06 False
SelectToList - Duration of single invocation 292.81 ns 125.57 ns 0.43 0.01 True
SelectToList - Duration of single invocation 803.53 ns 381.40 ns 0.47 0.14 True
All_AllElementsMatch - Duration of single invocation 550.64 ns 256.09 ns 0.47 0.29 True
WhereLast_LastElementMatches - Duration of single invocation 335.90 ns 260.24 ns 0.77 0.07 True
LastWithPredicate_FirstElementMatches - Duration of single invocation 11.85 ns 8.00 ns 0.68 0.00 True
Append - Duration of single invocation 3.40 μs 2.05 μs 0.60 0.15 True
Concat_TenTimes - Duration of single invocation 10.05 μs 5.09 μs 0.51 0.16 True
Skip_One - Duration of single invocation 1.01 μs 308.65 ns 0.31 0.13 True
Where - Duration of single invocation 674.45 ns 280.73 ns 0.42 0.04 True
FirstWithPredicate_LastElementMatches - Duration of single invocation 659.86 ns 257.69 ns 0.39 0.37 False
SelectToArray - Duration of single invocation 279.70 ns 107.31 ns 0.38 0.08 True
WhereLast_LastElementMatches - Duration of single invocation 605.35 ns 310.63 ns 0.51 0.15 False
Range - Duration of single invocation 492.52 ns 253.21 ns 0.51 0.15 True
EmptyTakeSelectToArray - Duration of single invocation 16.94 ns 13.61 ns 0.80 0.13 False
Select - Duration of single invocation 982.00 ns 316.68 ns 0.32 0.15 True
Except - Duration of single invocation 3.03 μs 2.44 μs 0.81 0.01 True
AnyWithPredicate_LastElementMatches - Duration of single invocation 555.08 ns 260.73 ns 0.47 0.21 False
SingleWithPredicate_FirstElementMatches - Duration of single invocation 552.52 ns 256.79 ns 0.46 0.35 False
AppendPrepend - Duration of single invocation 3.10 μs 2.11 μs 0.68 0.21 False
SequenceEqual - Duration of single invocation 982.40 ns 314.57 ns 0.32 0.29 True
SelectToArray - Duration of single invocation 838.25 ns 412.68 ns 0.49 0.02 True
ToDictionary - Duration of single invocation 2.19 μs 1.79 μs 0.82 0.04 True
WhereFirst_LastElementMatches - Duration of single invocation 329.86 ns 257.33 ns 0.78 0.11 False
WhereSingleOrDefault_LastElementMatches - Duration of single invocation 593.24 ns 303.98 ns 0.51 0.17 False
Intersect - Duration of single invocation 3.54 μs 2.74 μs 0.78 0.04 True
Where - Duration of single invocation 1.00 μs 319.47 ns 0.32 0.14 True
FirstWithPredicate_LastElementMatches - Duration of single invocation 993.27 ns 288.00 ns 0.29 0.17 True
Where - Duration of single invocation 714.67 ns 323.85 ns 0.45 0.03 True
Concat_Once - Duration of single invocation 1.77 μs 917.98 ns 0.52 0.19 True
SingleWithPredicate_FirstElementMatches - Duration of single invocation 915.05 ns 231.83 ns 0.25 0.15 True
Average - Duration of single invocation 489.11 ns 256.47 ns 0.52 0.41 False
Sum - Duration of single invocation 469.71 ns 256.79 ns 0.55 0.14 False
SequenceEqual - Duration of single invocation 21.80 ns 17.62 ns 0.81 0.26 False
Aggregate_Seed - Duration of single invocation 586.60 ns 260.00 ns 0.44 0.24 True
ToList - Duration of single invocation 632.78 ns 354.10 ns 0.56 0.03 True
LastWithPredicate_FirstElementMatches - Duration of single invocation 1.15 μs 363.72 ns 0.32 0.16 True
WhereAny_LastElementMatches - Duration of single invocation 596.22 ns 312.90 ns 0.52 0.13 False
WhereSingleOrDefault_LastElementMatches - Duration of single invocation 364.61 ns 254.32 ns 0.70 0.07 True
AnyWithPredicate_LastElementMatches - Duration of single invocation 910.13 ns 291.14 ns 0.32 0.14 True
Min - Duration of single invocation 439.47 ns 253.26 ns 0.58 0.16 True
Select - Duration of single invocation 619.20 ns 267.18 ns 0.43 0.17 True
Contains_ElementNotFound - Duration of single invocation 12.03 ns 10.37 ns 0.86 0.08 False
SingleWithPredicate_LastElementMatches - Duration of single invocation 550.74 ns 259.21 ns 0.47 0.38 False
Prepend - Duration of single invocation 2.63 μs 2.02 μs 0.77 0.27 False
Max - Duration of single invocation 412.43 ns 254.10 ns 0.62 0.27 False
WhereFirst_LastElementMatches - Duration of single invocation 619.02 ns 307.67 ns 0.50 0.16 False
TakeLastHalf - Duration of single invocation 696.30 ns 378.38 ns 0.54 0.13 True
SelectToArray - Duration of single invocation 302.53 ns 113.71 ns 0.38 0.10 True
SelectToArray - Duration of single invocation 244.79 ns 104.17 ns 0.43 0.04 True
CastToSameType - Duration of single invocation 414.34 ns 263.36 ns 0.64 0.18 True
ToList - Duration of single invocation 39.63 ns 35.06 ns 0.88 0.04 False
CastToBaseClass - Duration of single invocation 1.98 μs 1.66 μs 0.84 0.24 False
WhereSelect - Duration of single invocation 1.17 μs 360.87 ns 0.31 0.13 True
Take_All - Duration of single invocation 977.46 ns 322.82 ns 0.33 0.17 True
SingleWithPredicate_LastElementMatches - Duration of single invocation 554.59 ns 260.83 ns 0.47 0.33 False
ToDictionary - Duration of single invocation 935.15 ns 795.40 ns 0.85 0.01 True
WhereAny_LastElementMatches - Duration of single invocation 330.77 ns 260.23 ns 0.79 0.05 True
SkipHalfTakeHalf - Duration of single invocation 703.72 ns 300.78 ns 0.43 0.10 True
WhereSingle_LastElementMatches - Duration of single invocation 598.28 ns 303.50 ns 0.51 0.15 False
SelectToArray - Duration of single invocation 445.96 ns 120.05 ns 0.27 0.18 True
OrderByDescending - Duration of single invocation 3.69 μs 2.21 μs 0.60 0.01 True
ToDictionary - Duration of single invocation 869.92 ns 776.74 ns 0.89 0.01 True
Distinct - Duration of single invocation 2.37 μs 1.86 μs 0.78 0.06 False
WhereSelect - Duration of single invocation 881.62 ns 377.19 ns 0.43 0.11 True
Select - Duration of single invocation 978.31 ns 315.50 ns 0.32 0.17 True
SingleWithPredicate_LastElementMatches - Duration of single invocation 914.57 ns 236.50 ns 0.26 0.18 True
WhereSelect - Duration of single invocation 864.43 ns 304.34 ns 0.35 0.03 True
FirstWithPredicate_LastElementMatches - Duration of single invocation 2.48 μs 1.38 μs 0.56 0.03 True
SelectToList - Duration of single invocation 258.16 ns 107.74 ns 0.42 0.02 True
GroupBy - Duration of single invocation 2.43 μs 1.74 μs 0.71 0.05 True
ElementAt - Duration of single invocation 6.47 ns 4.91 ns 0.76 0.17 False
Zip - Duration of single invocation 1.48 μs 729.49 ns 0.49 0.13 True
SelectToList - Duration of single invocation 452.65 ns 141.33 ns 0.31 0.16 True
Select - Duration of single invocation 817.98 ns 321.25 ns 0.39 0.11 True
Repeat - Duration of single invocation 437.90 ns 255.89 ns 0.58 0.19 False
FirstWithPredicate_LastElementMatches - Duration of single invocation 686.35 ns 261.41 ns 0.38 0.34 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.ElementAt(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 118.32124755749194 < 127.61469876847727.
IsChangePoint: Marked as a change because one of 4/17/2023 7:48:00 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 43.096707758095874 (T) = (0 -118.30916802448641) / Math.Sqrt((222.39387415958842 / (299)) + (0.0016843154503335426 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23907375220264027 = (155.48046655895226 - 118.30916802448641) / 155.48046655895226 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Aggregate(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 259.862382141712 < 601.4472422201263.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 135.62211629794348 (T) = (0 -260.05652425232665) / Math.Sqrt((2137.107121661637 / (299)) + (0.3111233889751216 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5827389445933799 = (623.2465764122226 - 260.05652425232665) / 623.2465764122226 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.OrderBy(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.6114903779980043 < 2.7849452005729423.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 165.05468132444986 (T) = (0 -1629.2034390918056) / Math.Sqrt((4654.521514111832 / (299)) + (665.70279257259 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.45291337557835176 = (2977.962476809071 - 1629.2034390918056) / 2977.962476809071 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: Array)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 259.354292759471 < 524.9133794788215.
IsChangePoint: Marked as a change because one of 4/18/2023 3:11:52 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 125.6555955717328 (T) = (0 -258.94093376521573) / Math.Sqrt((2530.3704763809956 / (299)) + (1.5120725811236813 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5870076506138547 = (626.9872411682561 - 258.94093376521573) / 626.9872411682561 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToList(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 111.29912372890836 < 272.85333263395967.
IsChangePoint: Marked as a change because one of 4/25/2023 9:59:52 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 175.89072151128198 (T) = (0 -114.95591267359109) / Math.Sqrt((205.66032764827952 / (299)) + (6.603681458217896 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6259130119189625 = (307.2972766662723 - 114.95591267359109) / 307.2972766662723 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 255.21710278566874 < 546.7412639100675.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 130.7792825393421 (T) = (0 -255.0494448089873) / Math.Sqrt((2346.800162013391 / (299)) + (0.07313702605832959 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5896682493535231 = (621.5688754456787 - 255.0494448089873) / 621.5688754456787 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.OrderByThenBy(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.9230115976331363 < 4.063680699523268.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 57.089006205847696 (T) = (0 -2109.423546525805) / Math.Sqrt((5464.930793117704 / (299)) + (19312.094486240007 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.512078776899029 = (4323.2871345899175 - 2109.423546525805) / 4323.2871345899175 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 258.8852374023715 < 393.0744198211516.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 193.8681472705509 (T) = (0 -258.6104513330761) / Math.Sqrt((174.8956894102248 / (299)) + (3.074365742773839 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4045596864438531 = (434.31800878341187 - 258.6104513330761) / 434.31800878341187 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 252.39930437892997 < 530.0100681741035.
IsChangePoint: Marked as a change because one of 4/17/2023 1:30:24 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 252.5332388682048 (T) = (0 -252.65317629326566) / Math.Sqrt((555.8789266328762 / (299)) + (0.13477920294510046 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5774610118192369 = (597.940505753235 - 252.65317629326566) / 597.940505753235 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 935.5595647694579 < 1.1480763090587889.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 98.44790541188962 (T) = (0 -915.7253331228403) / Math.Sqrt((788.2738607540937 / (299)) + (124.75451696619372 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27326384557415806 = (1260.0519838541807 - 915.7253331228403) / 1260.0519838541807 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IOrderedEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.395374734466277 < 2.3233298966126994.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 188.08183652948327 (T) = (0 -1377.3163412174997) / Math.Sqrt((3581.7363066753846 / (299)) + (287.0004835419007 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.443489539284443 = (2474.9154570186456 - 1377.3163412174997) / 2474.9154570186456 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 255.08742037691013 < 319.2064219725873.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 153.26483465948553 (T) = (0 -256.03649893492485) / Math.Sqrt((82.2685933606798 / (299)) + (0.7019988779191199 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2556351625621455 = (343.9664074087874 - 256.03649893492485) / 343.9664074087874 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.ToArray(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 427.88066421995 < 673.524532069954.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 217.66179569386142 (T) = (0 -423.1109211512145) / Math.Sqrt((316.35964738141877 / (299)) + (11.95262603347797 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.41975547452576367 = (729.194163108052 - 423.1109211512145) / 729.194163108052 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Reverse(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 569.2219831532685 < 1.1138860530857464.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 285.53416482993873 (T) = (0 -569.8604994904458) / Math.Sqrt((1395.2179453462493 / (299)) + (0.8761334490505756 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5215668825000882 = (1191.0975194783643 - 569.8604994904458) / 1191.0975194783643 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.ToArray(input: ICollection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 31.645121882435873 < 37.45138404942045.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 19.18383908734127 (T) = (0 -32.01397469031462) / Math.Sqrt((2.0573009570632106 / (299)) + (1.4755742130028426 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17212673867602526 = (38.670139725392666 - 32.01397469031462) / 38.670139725392666 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToList(input: Array)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 125.5654285978634 < 273.648308979315.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 137.82296453835525 (T) = (0 -113.17320401846064) / Math.Sqrt((102.95163116059034 / (299)) + (19.69279540744637 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6241288260221981 = (301.09572601900135 - 113.17320401846064) / 301.09572601900135 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToList(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 381.3975309098681 < 763.3596341101132.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 189.89159056846557 (T) = (0 -379.12628010763126) / Math.Sqrt((953.0111519067428 / (299)) + (35.493983138714846 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5492281022190713 = (841.0601503199372 - 379.12628010763126) / 841.0601503199372 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.All_AllElementsMatch(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 256.0893774531154 < 537.0548880853298.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 212.91994629996788 (T) = (0 -256.111890218383) / Math.Sqrt((732.2275732539973 / (299)) + (0.03311868660461471 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5655322497635729 = (589.4842369290078 - 256.111890218383) / 589.4842369290078 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 260.23992950866614 < 318.4583630971403.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 158.9776853991393 (T) = (0 -260.34339480548584) / Math.Sqrt((85.90191087134203 / (299)) + (0.05322170695042722 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24791158586375497 = (346.1606240863101 - 260.34339480548584) / 346.1606240863101 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input: IList)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.004566059739675 < 11.25798832600367.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 358.70125230934667 (T) = (0 -8.006361816780803) / Math.Sqrt((0.03536314696054413 / (299)) + (6.330365494279183E-06 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.32806353502636104 = (11.915355445242735 - 8.006361816780803) / 11.915355445242735 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Append(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.0518261057951763 < 3.125273397496938.
IsChangePoint: Marked as a change because one of 4/17/2023 1:30:24 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 139.7960228282264 (T) = (0 -2041.2010932408507) / Math.Sqrt((19488.073474605964 / (299)) + (280.76768105122534 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3894915593887279 = (3343.444508641185 - 2041.2010932408507) / 3343.444508641185 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Concat_TenTimes(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.093399951550388 < 8.420848241836351.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 136.3104648607412 (T) = (0 -5195.819552406658) / Math.Sqrt((154077.77243066672 / (299)) + (5269.464547108949 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4432129360104764 = (9331.78927537803 - 5195.819552406658) / 9331.78927537803 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Skip_One(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 308.6474770094121 < 903.8551973650679.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 345.8439075165551 (T) = (0 -309.2047283249624) / Math.Sqrt((1046.1911894438804 / (299)) + (0.4094602026870715 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.677585684063215 = (959.0291529907976 - 309.2047283249624) / 959.0291529907976 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Where(input: Array)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 280.7278195522857 < 638.4688860849553.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 673.2849529033056 (T) = (0 -279.43546153742994) / Math.Sqrt((65.84657168389941 / (299)) + (1.621666237500537 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5859488241416827 = (674.88145869449 - 279.43546153742994) / 674.88145869449 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 257.6924353527514 < 522.8843603507694.
IsChangePoint: Marked as a change because one of 3/7/2023 11:50:15 AM, 3/14/2023 8:42:09 PM, 4/17/2023 7:48:00 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 107.10922479972865 (T) = (0 -256.93662558599465) / Math.Sqrt((4033.8192736033534 / (299)) + (0.15033604736204828 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6050282064204603 = (650.5189225221284 - 256.93662558599465) / 650.5189225221284 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToArray(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 107.30683392380638 < 267.23791886259716.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 224.33329645084567 (T) = (0 -108.8855276961311) / Math.Sqrt((78.0620604354388 / (299)) + (3.8799769558281128 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6064735519804789 = (276.69176555759674 - 108.8855276961311) / 276.69176555759674 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 310.6343449446375 < 583.2426237044276.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 131.4450953560276 (T) = (0 -311.10733639874593) / Math.Sqrt((1824.249468845002 / (299)) + (13.606520781609 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5304261744103443 = (662.5312558852287 - 311.10733639874593) / 662.5312558852287 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Range

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 253.21467363405577 < 423.065683298949.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 157.76333892450157 (T) = (0 -253.22588748738866) / Math.Sqrt((426.16401652481676 / (299)) + (0.5209859782365551 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4299314665869683 = (444.20253468703373 - 253.22588748738866) / 444.20253468703373 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.EmptyTakeSelectToArray

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.611134207644863 < 16.309198632782916.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/29/2023 4:36:15 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 56.83153170252132 (T) = (0 -13.549024062201166) / Math.Sqrt((0.8901613874604316 / (299)) + (0.031524018981379 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23564402374288618 = (17.72606544995934 - 13.549024062201166) / 17.72606544995934 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Select(input: IList)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 316.6819101990883 < 935.2165227481905.
IsChangePoint: Marked as a change because one of 4/17/2023 1:30:24 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 190.78978916736588 (T) = (0 -316.0189181377183) / Math.Sqrt((4601.306959411261 / (299)) + (1.4038063853679021 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7038486016818739 = (1067.0856863496908 - 316.0189181377183) / 1067.0856863496908 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Except(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.4425640542577245 < 2.881206585605057.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 39.62795789731742 (T) = (0 -2473.555079158379) / Math.Sqrt((1825.3294551440217 / (299)) + (2659.920545022269 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1886775935676009 = (3048.7942395616315 - 2473.555079158379) / 3048.7942395616315 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: Array)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 260.7261331229061 < 563.5484271958728.
IsChangePoint: Marked as a change because one of 4/17/2023 7:48:00 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 128.95728240178454 (T) = (0 -259.03539733826926) / Math.Sqrt((2282.504907965567 / (299)) + (0.7959143139832682 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.580007250822051 = (616.7615937305553 - 259.03539733826926) / 616.7615937305553 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 256.79197089574666 < 524.8217627972814.
IsChangePoint: Marked as a change because one of 4/17/2023 1:30:24 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 123.43155042104814 (T) = (0 -256.48883833081874) / Math.Sqrt((2689.9407828935114 / (299)) + (0.41821073945468706 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5911694167288487 = (627.3719453143407 - 256.48883833081874) / 627.3719453143407 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.AppendPrepend(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.1148517097554764 < 2.8755716823141007.
IsChangePoint: Marked as a change because one of 4/11/2023 10:53:42 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 15.996129892232727 (T) = (0 -2170.8137257530657) / Math.Sqrt((16970.56708580377 / (299)) + (51388.74352033539 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.31815363621870274 = (3183.728536314899 - 2170.8137257530657) / 3183.728536314899 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SequenceEqual(input1: IEnumerable, input2: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 314.5670105591733 < 933.75316040196.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 171.54397283242804 (T) = (0 -314.7295798079842) / Math.Sqrt((5857.214893313556 / (299)) + (0.22516450828257387 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7070419745790453 = (1074.3162927717913 - 314.7295798079842) / 1074.3162927717913 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 412.67672754441634 < 805.1182883590151.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 188.5537011950442 (T) = (0 -422.1861188389622) / Math.Sqrt((519.827536109364 / (299)) + (53.48419289887908 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5193378618419456 = (878.3427803505011 - 422.1861188389622) / 878.3427803505011 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.ToDictionary(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.7861901222062475 < 2.075743802458559.
IsChangePoint: Marked as a change because one of 4/26/2023 6:27:29 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 31.81867638663481 (T) = (0 -1712.3716787940978) / Math.Sqrt((1889.4136351248126 / (299)) + (3394.0235463812155 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23304216730921598 = (2232.6803453932234 - 1712.3716787940978) / 2232.6803453932234 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 257.3254700785412 < 313.98185104812353.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 136.5300061444856 (T) = (0 -257.20712444674183) / Math.Sqrt((109.43248307510446 / (299)) + (0.21249530340056444 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2471149694330943 = (341.62868698965974 - 257.20712444674183) / 341.62868698965974 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 303.9768336575232 < 631.5329370464244.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 100.66704552056531 (T) = (0 -307.5651115630213) / Math.Sqrt((3432.914527787674 / (299)) + (7.323690869727411 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5318177544913291 = (656.9345901377737 - 307.5651115630213) / 656.9345901377737 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Intersect(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.7435670244854506 < 3.3635138124416732.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 32.9574265165538 (T) = (0 -2845.345583376967) / Math.Sqrt((7465.291547974702 / (299)) + (7282.120605029704 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21886282590738668 = (3642.5683961106897 - 2845.345583376967) / 3642.5683961106897 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Where(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 319.4677780089462 < 0.9781762135213592.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 274.07661010906355 (T) = (0 -320.2524820085937) / Math.Sqrt((2050.4790682520793 / (299)) + (4.399177729163919 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6965819065650851 = (1055.482480899874 - 320.2524820085937) / 1055.482480899874 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 288.002767184448 < 917.1382980097551.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 318.36263044939693 (T) = (0 -288.6620440843344) / Math.Sqrt((1175.6211449731297 / (299)) + (9.006226112408234 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7034190597126637 = (973.2993758960713 - 288.6620440843344) / 973.2993758960713 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Where(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 323.84537723158974 < 674.609224906737.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 239.89170621625573 (T) = (0 -321.52200971450475) / Math.Sqrt((510.7686588686845 / (299)) + (14.339697455913623 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5557539009912148 = (723.7475138935243 - 321.52200971450475) / 723.7475138935243 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Concat_Once(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 917.9756742245902 < 1.570023343045627.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 130.8018799726301 (T) = (0 -917.9860753533594) / Math.Sqrt((8026.242123297144 / (299)) + (132.6644455289131 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.46446415335950264 = (1714.1449654808991 - 917.9860753533594) / 1714.1449654808991 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 231.83027741552917 < 919.3278486156119.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 375.1278960751145 (T) = (0 -232.15771223855168) / Math.Sqrt((1108.4695614745187 / (299)) + (0.3543845608337515 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.757433391858807 = (957.0885045455947 - 232.15771223855168) / 957.0885045455947 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Average(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 256.46684736660353 < 433.4413656693386.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 84.4831601681137 (T) = (0 -256.2508119639829) / Math.Sqrt((1649.0404117187218 / (299)) + (0.6383713218191579 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4374740927122523 = (455.53601824227354 - 256.2508119639829) / 455.53601824227354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Sum(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 256.7933391711903 < 444.34808906804517.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 140.17715047482935 (T) = (0 -256.97964415148607) / Math.Sqrt((555.1651592009415 / (299)) + (0.19470752756710077 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4273523510807889 = (448.7570055277406 - 256.97964415148607) / 448.7570055277406 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SequenceEqual(input1: Array, input2: Array)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.621924075106467 < 20.756026829805936.
IsChangePoint: Marked as a change because one of 3/1/2023 11:57:24 AM, 3/22/2023 10:50:22 PM, 3/23/2023 6:11:55 PM, 5/19/2023 10:55:27 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 25.81608884524079 (T) = (0 -17.65708966138886) / Math.Sqrt((7.528607650256241 / (299)) + (0.019565842737612475 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.19409958674867425 = (21.909766232968007 - 17.65708966138886) / 21.909766232968007 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Aggregate_Seed(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 259.9974015094515 < 524.3916090278284.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 170.1319682264722 (T) = (0 -260.00440824979535) / Math.Sqrt((1176.6515779801182 / (299)) + (0.5329264478580236 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5661228510330909 = (599.2581284100429 - 260.00440824979535) / 599.2581284100429 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.ToList(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 354.09899020923865 < 596.2905695690172.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 234.48168121798045 (T) = (0 -351.46550728939377) / Math.Sqrt((341.4649883234561 / (299)) + (6.71066367551654 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.46210797991644115 = (653.4127560300957 - 351.46550728939377) / 653.4127560300957 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input: IOrderedEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 363.7241702514981 < 1.0910588270054813.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 386.71141601129966 (T) = (0 -363.52268020819633) / Math.Sqrt((1284.2353393800922 / (299)) + (0.15154153388868707 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6882455454752265 = (1166.0544859329639 - 363.52268020819633) / 1166.0544859329639 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 312.89902010474736 < 566.6694603675671.
IsChangePoint: Marked as a change because one of 4/25/2023 12:29:35 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 131.77298538720984 (T) = (0 -314.23877093268135) / Math.Sqrt((1907.6456212691448 / (299)) + (2.3666968867017286 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5178888241933524 = (651.7973170958143 - 314.23877093268135) / 651.7973170958143 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 254.31656054505174 < 324.7243671544194.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 141.57784407474892 (T) = (0 -255.16740243653783) / Math.Sqrt((93.06738120424349 / (299)) + (1.0077146349799446 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2570338564168889 = (343.4441860377906 - 255.16740243653783) / 343.4441860377906 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 291.1371201986796 < 864.8400177753113.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 314.63264514755525 (T) = (0 -284.13898949977334) / Math.Sqrt((841.9353715890735 / (299)) + (22.700830313307176 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.702833792215963 = (956.1618449775723 - 284.13898949977334) / 956.1618449775723 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Min(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 253.26403381685074 < 392.9513929972053.
IsChangePoint: Marked as a change because one of 4/25/2023 9:59:52 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 191.46139038187138 (T) = (0 -252.73546414124303) / Math.Sqrt((277.5385623489045 / (299)) + (0.04131090005851824 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4223362400197778 = (437.51310303747647 - 252.73546414124303) / 437.51310303747647 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Select(input: Array)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 267.18227549651067 < 562.3606597314738.
IsChangePoint: Marked as a change because one of 4/22/2023 6:52:03 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 140.0207250598782 (T) = (0 -268.9066930028224) / Math.Sqrt((2286.258153928523 / (299)) + (5.17442496738679 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5962617349576461 = (666.0421275021156 - 268.9066930028224) / 666.0421275021156 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: ICollection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.368283719163749 < 11.139451996629036.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/10/2023 1:33:42 PM, 4/25/2023 3:46:39 AM, 5/11/2023 5:02:01 PM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.822415170843326 (T) = (0 -10.61501924434891) / Math.Sqrt((0.18205025575502098 / (299)) + (0.07585300694219257 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.06596493614978184 = (11.364690315363935 - 10.61501924434891) / 11.364690315363935 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: Array)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 259.20769915929617 < 523.4185036190041.
IsChangePoint: Marked as a change because one of 4/18/2023 3:11:52 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 103.80112778665399 (T) = (0 -259.56859427858126) / Math.Sqrt((3889.3207724078975 / (299)) + (0.5065743494555172 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5909090382976365 = (634.500926636047 - 259.56859427858126) / 634.500926636047 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Prepend(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.021899970508369 < 2.6844694445662562.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 56.705330601184244 (T) = (0 -1992.9181601618327) / Math.Sqrt((32242.15680896304 / (299)) + (1557.3413083993298 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3003505879434102 = (2848.4525618391276 - 1992.9181601618327) / 2848.4525618391276 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Max(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 254.10498718746612 < 391.7825620412487.
IsChangePoint: Marked as a change because one of 4/25/2023 9:59:52 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 144.49831850193084 (T) = (0 -254.2973824688409) / Math.Sqrt((501.56809536940466 / (299)) + (0.04728213016914632 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4242121265901222 = (441.6511604575213 - 254.2973824688409) / 441.6511604575213 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 307.66707592154734 < 562.5172419413236.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 131.80498863205287 (T) = (0 -306.60021852110185) / Math.Sqrt((1887.9590741258642 / (299)) + (8.008796833748344 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5308932856412932 = (653.583095565452 - 306.60021852110185) / 653.583095565452 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 378.3838861048775 < 662.9606940120731.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 201.77124206080603 (T) = (0 -378.5446093592994) / Math.Sqrt((917.4127359595868 / (299)) + (0.15230453286522047 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.48332197762783335 = (732.6508830805874 - 378.5446093592994) / 732.6508830805874 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Array)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 113.71165284855147 < 259.37906158890803.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 119.3599599946886 (T) = (0 -109.15980432805881) / Math.Sqrt((181.97567548591218 / (299)) + (14.708762445768832 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5905622073616441 = (266.6090094533027 - 109.15980432805881) / 266.6090094533027 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Range)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 104.17101703926036 < 233.1720500273893.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 178.47051174243904 (T) = (0 -103.4423331463861) / Math.Sqrt((165.54416229624408 / (299)) + (2.2982822760580124 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5958883434306428 = (255.9746334083596 - 103.4423331463861) / 255.9746334083596 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.CastToSameType(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 263.36415220582415 < 393.85872293541684.
IsChangePoint: Marked as a change because one of 4/25/2023 9:59:52 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 163.1601294384418 (T) = (0 -263.1959365791705) / Math.Sqrt((304.3590859154671 / (299)) + (2.4256109710588647 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.40489002852362255 = (442.2643699385869 - 263.1959365791705) / 442.2643699385869 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.ToList(input: ICollection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 35.061350445769335 < 40.90264979970799.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 39.25953603975393 (T) = (0 -35.92081662563716) / Math.Sqrt((2.3829929511474144 / (299)) + (0.42714304978066703 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1808901858510179 = (43.85348094376974 - 35.92081662563716) / 43.85348094376974 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.CastToBaseClass(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.66279274792721 < 1.8807820247630131.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 25.363465911015417 (T) = (0 -1710.4986130564284) / Math.Sqrt((29460.28370221631 / (299)) + (811.806041189567 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1583438296783925 = (2032.300924500827 - 1710.4986130564284) / 2032.300924500827 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereSelect(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 360.86643483490474 < 1.1141864815166715.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 208.62751261116554 (T) = (0 -362.8733814891175) / Math.Sqrt((5353.928321121901 / (299)) + (8.994344116379859 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7125954676316382 = (1262.587539934925 - 362.8733814891175) / 1262.587539934925 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Take_All(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 322.82453601915415 < 892.805875483621.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 194.70611795027142 (T) = (0 -324.59939642519714) / Math.Sqrt((3030.517129210973 / (299)) + (1.205126382546857 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.657342632408421 = (947.3002104308883 - 324.59939642519714) / 947.3002104308883 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 260.8297825805197 < 525.9638719979486.
IsChangePoint: Marked as a change because one of 4/17/2023 1:30:24 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 122.37522903129945 (T) = (0 -260.2038543026014) / Math.Sqrt((2678.9616765478518 / (299)) + (0.43880688164741133 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5851320373929512 = (627.1967897146571 - 260.2038543026014) / 627.1967897146571 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.ToDictionary(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 795.4045345022788 < 885.9512183664568.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 24.024802976859878 (T) = (0 -800.7489010020345) / Math.Sqrt((146.82347502176808 / (299)) + (387.4667704294832 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1417354893367565 = (932.986149436888 - 800.7489010020345) / 932.986149436888 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 260.23307264745625 < 314.9680314730387.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 162.21761160924368 (T) = (0 -260.3542880064127) / Math.Sqrt((73.00964910538481 / (299)) + (0.22241355361824577 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24155550503190354 = (343.2740164029595 - 260.3542880064127) / 343.2740164029595 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SkipHalfTakeHalf(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 300.7773923738685 < 614.8328194914154.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 220.1626245008882 (T) = (0 -299.9806836064028) / Math.Sqrt((657.8434726763755 / (299)) + (1.2388553053094167 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5265029095760236 = (633.542823542581 - 299.9806836064028) / 633.542823542581 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 303.50038252433507 < 567.5770068276615.
IsChangePoint: Marked as a change because one of 4/17/2023 7:48:00 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 145.82045135521594 (T) = (0 -306.11227519669546) / Math.Sqrt((1451.1029595622585 / (299)) + (8.64890524361965 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5280894051894621 = (648.6658247619829 - 306.11227519669546) / 648.6658247619829 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IList)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 120.04542628891888 < 421.9119392523736.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 188.62436790952086 (T) = (0 -121.19589024138942) / Math.Sqrt((569.7596392396053 / (299)) + (11.37019232813258 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.721840073252872 = (435.70578860400406 - 121.19589024138942) / 435.70578860400406 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.OrderByDescending(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.214117374475277 < 3.5245530156218874.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 172.18408383552904 (T) = (0 -2205.5354149369928) / Math.Sqrt((9302.941529793648 / (299)) + (822.0192225309421 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.43126834044889645 = (3877.9895191307064 - 2205.5354149369928) / 3877.9895191307064 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.ToDictionary(input: Array)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 776.7391210056466 < 825.6984991160401.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 38.625834419625065 (T) = (0 -742.6420687527861) / Math.Sqrt((190.88695378645016 / (299)) + (186.52426948240213 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16760180030033556 = (892.1716421548449 - 742.6420687527861) / 892.1716421548449 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Distinct(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.860028105509872 < 2.256032486814569.
IsChangePoint: Marked as a change because one of 4/22/2023 7:41:29 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 25.71519991294967 (T) = (0 -1887.3779223318243) / Math.Sqrt((3497.756397781335 / (299)) + (4782.809094915962 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20977306560001618 = (2388.3998889064733 - 1887.3779223318243) / 2388.3998889064733 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereSelect(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 377.18673486777124 < 833.3866880368807.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 165.59007486498527 (T) = (0 -372.44882711570455) / Math.Sqrt((2413.3722634067085 / (299)) + (17.870762873212737 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5774269109652028 = (881.3832134138454 - 372.44882711570455) / 881.3832134138454 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Select(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 315.5000847314015 < 1.0071899565312499.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 181.75833269459486 (T) = (0 -314.81015131815536) / Math.Sqrt((5065.235727247369 / (299)) + (1.4632834737003575 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7045116817812804 = (1065.389499036418 - 314.81015131815536) / 1065.389499036418 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 236.5046672233582 < 915.6567143069238.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 322.9715543484293 (T) = (0 -236.6366130094111) / Math.Sqrt((1495.7127649092893 / (299)) + (0.11822095708855533 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7534139854220411 = (959.6513955360505 - 236.6366130094111) / 959.6513955360505 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.WhereSelect(input: Array)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 304.34044947393767 < 795.2806208247108.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 567.5340381522524 (T) = (0 -302.3886322569624) / Math.Sqrt((255.38558751338923 / (299)) + (1.3789981114152043 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.647777823374402 = (858.5167326882795 - 302.3886322569624) / 858.5167326882795 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IOrderedEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3764100356012121 < 2.321111782434363.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 185.53483113450585 (T) = (0 -1365.8105757338244) / Math.Sqrt((4390.030875369022 / (299)) + (292.83478002398084 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.45313866525526686 = (2497.5446040107495 - 1365.8105757338244) / 2497.5446040107495 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToList(input: Range)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 107.73521461545586 < 245.0731390437004.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 208.96608353679193 (T) = (0 -106.36418142575553) / Math.Sqrt((199.59783997569838 / (299)) + (1.1489930099387218 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6307424826122401 = (288.0487909310774 - 106.36418142575553) / 288.0487909310774 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.GroupBy(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.7376877182770663 < 2.2825635702262117.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 26.97200739282229 (T) = (0 -1739.7195341054362) / Math.Sqrt((2861.6486906787377 / (299)) + (6560.3990829279055 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26011765557262073 = (2351.3461933624944 - 1739.7195341054362) / 2351.3461933624944 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.ElementAt(input: IList)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.90724831633445 < 6.15900558112701.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 102.3660806860077 (T) = (0 -4.912361341040692) / Math.Sqrt((0.09216294557341963 / (299)) + (0.0001772066626410155 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27212199395004943 = (6.748880032382215 - 4.912361341040692) / 6.748880032382215 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Zip(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 729.4879125495489 < 1.4078124904417244.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 269.68514281240573 (T) = (0 -728.0255799436126) / Math.Sqrt((2079.585536822587 / (299)) + (24.0713838224198 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5236523145699971 = (1528.3491496057507 - 728.0255799436126) / 1528.3491496057507 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToList(input: IList)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 141.33391493690388 < 405.8593642236589.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 218.8449592227436 (T) = (0 -140.81931511831274) / Math.Sqrt((602.44660211401 / (299)) + (3.915036664332689 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7028304420172131 = (473.86857548332557 - 140.81931511831274) / 473.86857548332557 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Select(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 321.2489142066933 < 780.3489941248779.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 318.9212471681601 (T) = (0 -324.87581584061655) / Math.Sqrt((572.8810347931619 / (299)) + (7.210190537814071 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.606765385621531 = (826.1628146700727 - 324.87581584061655) / 826.1628146700727 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Repeat

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 255.88906788399208 < 363.8772407711079.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 138.75413508519685 (T) = (0 -256.1901590324574) / Math.Sqrt((279.5941440674688 / (299)) + (0.13147778668688911 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.34493241985959616 = (391.08966280631216 - 256.1901590324574) / 391.08966280631216 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: Array)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 261.41098186406657 < 599.8126195386378.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 101.21110691786194 (T) = (0 -262.2846876857071) / Math.Sqrt((4393.628298271526 / (299)) + (3.373678348733908 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5987512850353827 = (653.671097012325 - 262.2846876857071) / 653.671097012325 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in GuardedDevirtualization.ThreeClassVirtual

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Call - Duration of single invocation 6.04 ns 1.52 ns 0.25 0.02 True
Call - Duration of single invocation 6.35 ns 1.63 ns 0.26 0.02 True
Call - Duration of single invocation 7.86 ns 3.94 ns 0.50 0.10 False
Call - Duration of single invocation 6.35 ns 1.51 ns 0.24 0.02 True
Call - Duration of single invocation 6.34 ns 1.35 ns 0.21 0.02 True
Call - Duration of single invocation 6.04 ns 1.53 ns 0.25 0.03 True
Call - Duration of single invocation 1.68 ns 0.49 ns 0.29 0.04 False
Call - Duration of single invocation 6.05 ns 1.49 ns 0.25 0.01 True
Call - Duration of single invocation 1.68 ns 0.49 ns 0.29 0.05 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'GuardedDevirtualization.ThreeClassVirtual*'

Payloads

Baseline
Compare

Histogram

GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.67 pD=0.33)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.5188297666055564 < 5.740924185885442.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 201.9075406942778 (T) = (0 -1.4873754373511483) / Math.Sqrt((0.0017073743567643895 / (299)) + (0.0066163385128224644 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7548768967748642 = (6.06787127684596 - 1.4873754373511483) / 6.06787127684596 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.33 pD=0.00)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.6253394740494034 < 6.034426075995877.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 92.72600462538175 (T) = (0 -1.7448794237094158) / Math.Sqrt((0.0024082466747760677 / (299)) + (0.032228265593387505 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7260463533782994 = (6.369250583909545 - 1.7448794237094158) / 6.369250583909545 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.33 pD=0.33)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.9411365702404955 < 7.467686117587985.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 36.123624724651684 (T) = (0 -4.04754550933307) / Math.Sqrt((0.246116295262123 / (299)) + (0.07798422545793002 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.42434276169421575 = (7.031172788247037 - 4.04754550933307) / 7.031172788247037 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.00 pD=0.33)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.5116850426645296 < 6.0338045778548395.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 86.76347268831303 (T) = (0 -1.6831284560217674) / Math.Sqrt((0.0024205246505649234 / (299)) + (0.03783689597837917 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7357919009662094 = (6.370465031832753 - 1.6831284560217674) / 6.370465031832753 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.33 pD=0.67)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.346623438541385 < 6.028583770277873.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 189.4524530602208 (T) = (0 -1.569548318276599) / Math.Sqrt((0.0023311081082780342 / (299)) + (0.008242554658369775 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.753573666830531 = (6.369239431880073 - 1.569548318276599) / 6.369239431880073 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.00 pD=0.67)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.526057720555003 < 5.743007012929348.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 82.01275814298869 (T) = (0 -1.5171208413241515) / Math.Sqrt((0.0017438570660959322 / (299)) + (0.039956980333415805 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7499897664322064 = (6.0682349665209365 - 1.5171208413241515) / 6.0682349665209365 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.00 pD=1.00)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.49077488062403557 < 1.5960964817622545.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 143.34949287281793 (T) = (0 -0.4929200019526415) / Math.Sqrt((0.017364428624111066 / (299)) + (3.4097776674483133E-06 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6895592634526223 = (1.5878070881893265 - 0.4929200019526415) / 1.5878070881893265 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.67 pD=0.00)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.4883560430356915 < 5.74005407319722.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 310.00866075611117 (T) = (0 -1.3070290391964556) / Math.Sqrt((0.0016999416228911158 / (299)) + (0.002992253186524272 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7846046609504583 = (6.068046991935301 - 1.3070290391964556) / 6.068046991935301 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=1.00 pD=0.00)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.494993017646312 < 1.596878683377016.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/9/2023 11:04:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 125.51191276165848 (T) = (0 -0.49470851898551926) / Math.Sqrt((0.020113331310178766 / (299)) + (1.2226615379157554E-06 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6755679860276187 = (1.5248449526551149 - 0.49470851898551926) / 1.5248449526551149 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsMatch - Duration of single invocation 348.84 ns 300.03 ns 0.86 0.21 False
IsMatch - Duration of single invocation 115.49 ns 77.91 ns 0.67 0.07 False
IsMatch - Duration of single invocation 122.96 ns 87.96 ns 0.72 0.09 False
IsMatch - Duration of single invocation 41.35 ns 38.15 ns 0.92 0.08 False
IsMatch - Duration of single invocation 78.85 ns 74.18 ns 0.94 0.07 False
IsMatch - Duration of single invocation 41.40 ns 35.06 ns 0.85 0.07 False
IsMatch - Duration of single invocation 171.81 ns 150.12 ns 0.87 0.05 False
IsMatch - Duration of single invocation 40.37 ns 37.58 ns 0.93 0.04 False
IsMatch - Duration of single invocation 39.82 ns 36.91 ns 0.93 0.06 False
IsMatch - Duration of single invocation 158.28 ns 136.71 ns 0.86 0.03 True
IsMatch - Duration of single invocation 166.13 ns 135.88 ns 0.82 0.13 False
IsMatch - Duration of single invocation 116.27 ns 99.43 ns 0.86 0.24 False
IsMatch - Duration of single invocation 375.80 ns 320.82 ns 0.85 0.16 False
IsMatch - Duration of single invocation 130.10 ns 98.36 ns 0.76 0.10 False
IsMatch - Duration of single invocation 110.98 ns 95.04 ns 0.86 0.24 False
IsMatch - Duration of single invocation 86.68 ns 59.86 ns 0.69 0.12 False
IsMatch - Duration of single invocation 159.70 ns 137.84 ns 0.86 0.03 True
IsMatch - Duration of single invocation 104.85 ns 98.33 ns 0.94 0.12 False
IsMatch - Duration of single invocation 179.72 ns 137.38 ns 0.76 0.18 False
IsMatch - Duration of single invocation 351.79 ns 282.92 ns 0.80 0.27 False
IsMatch - Duration of single invocation 118.00 ns 110.24 ns 0.93 0.08 False
IsMatch - Duration of single invocation 97.78 ns 77.71 ns 0.79 0.04 False
IsMatch - Duration of single invocation 271.98 ns 215.53 ns 0.79 0.04 False
IsMatch - Duration of single invocation 120.62 ns 86.73 ns 0.72 0.08 False
IsMatch - Duration of single invocation 44.45 ns 40.25 ns 0.91 0.07 False
IsMatch - Duration of single invocation 110.96 ns 90.97 ns 0.82 0.04 False
IsMatch - Duration of single invocation 179.67 ns 136.61 ns 0.76 0.03 True
IsMatch - Duration of single invocation 127.41 ns 95.20 ns 0.75 0.12 False
IsMatch - Duration of single invocation 210.13 ns 171.29 ns 0.82 0.02 False
IsMatch - Duration of single invocation 43.93 ns 41.08 ns 0.94 0.08 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 5, Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 300.0265496599164 < 319.9821733777542.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 9.847448883582418 (T) = (0 -306.88429546393087) / Math.Sqrt((253.85935833922406 / (299)) + (198.00567707061703 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11400561016013039 = (346.37272987630956 - 306.88429546393087) / 346.37272987630956 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 11, Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 77.90836481200523 < 108.07015467015093.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 44.092055689689985 (T) = (0 -79.69703135843469) / Math.Sqrt((32.71004587056148 / (299)) + (2.8169957816547364 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24007954706225212 = (104.87549196805656 - 79.69703135843469) / 104.87549196805656 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 12, Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 87.96066316714418 < 116.57488231121582.
IsChangePoint: Marked as a change because one of 4/19/2023 8:38:27 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 38.05350243383114 (T) = (0 -87.39086613304153) / Math.Sqrt((28.144169016195885 / (299)) + (3.8884554327545486 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2144907332468779 = (111.25376851920402 - 87.39086613304153) / 111.25376851920402 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 6, Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 38.15155104128748 < 39.21743781766817.
IsChangePoint: Marked as a change because one of 4/24/2023 8:06:00 AM, 5/1/2023 3:42:23 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 29.012052029289027 (T) = (0 -38.26760254243514) / Math.Sqrt((4.122200581547842 / (299)) + (0.041285454207512805 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08986621306753279 = (42.04612892288399 - 38.26760254243514) / 42.04612892288399 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 2, Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 74.17814512669894 < 84.92789579959407.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/28/2023 8:13:28 AM, 5/2/2023 3:18:41 PM, 5/19/2023 12:51:11 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 26.630863916568654 (T) = (0 -74.49259163493579) / Math.Sqrt((10.434084051345366 / (299)) + (0.6911072067017915 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.09233395643960368 = (82.07048414274962 - 74.49259163493579) / 82.07048414274962 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 11, Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 35.06231507586347 < 39.388196095568205.
IsChangePoint: Marked as a change because one of 3/6/2023 6:03:21 AM, 4/8/2023 3:09:32 AM, 5/5/2023 3:18:41 PM, 5/10/2023 5:00:45 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 47.000839598612416 (T) = (0 -34.2299314348965) / Math.Sqrt((3.9614116146190246 / (299)) + (0.19204322829097742 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18689277411733765 = (42.09768446927582 - 34.2299314348965) / 42.09768446927582 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 2, Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 150.12024055075977 < 161.3712368831836.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 32.793045050116945 (T) = (0 -151.73597257856912) / Math.Sqrt((80.53274356147536 / (299)) + (10.320597725727088 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18223638099557538 = (185.54991815764322 - 151.73597257856912) / 185.54991815764322 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 7, Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 37.57955964219092 < 38.365548994204964.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/1/2023 3:42:23 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 27.191163585916474 (T) = (0 -37.31068811683071) / Math.Sqrt((4.483061787543984 / (299)) + (0.08598730379158193 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09676168537773644 = (41.307689801040084 - 37.31068811683071) / 41.307689801040084 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 9, Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 36.906267156336206 < 38.10421545001757.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/20/2023 8:31:51 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 23.716159082038825 (T) = (0 -37.320775748959576) / Math.Sqrt((10.48561453300711 / (299)) + (0.12076348283110315 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11804036682465624 = (42.31574138443565 - 37.320775748959576) / 42.31574138443565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 13, Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 136.71404510268405 < 149.44111254085612.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 17.68900365979953 (T) = (0 -127.96963096877792) / Math.Sqrt((5.330333956796999 / (299)) + (37.290672221966496 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19017783234259714 = (158.021891817261 - 127.96963096877792) / 158.021891817261 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 1, Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 135.87735415474071 < 158.34615702378642.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 7.010199526681185 (T) = (0 -140.51020158020785) / Math.Sqrt((39.05692321109273 / (299)) + (164.1609911447521 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1512514561629409 = (165.54985878972263 - 140.51020158020785) / 165.54985878972263 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 11, Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 99.4273940345369 < 115.58406337579582.
IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 13.165683012309517 (T) = (0 -104.89572122493165) / Math.Sqrt((57.55420270763796 / (299)) + (25.90463638035044 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15649918607981098 = (124.35758151486117 - 104.89572122493165) / 124.35758151486117 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 3, Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 320.82029400259415 < 366.16408787421534.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 67.77165598736394 (T) = (0 -318.8474665243114) / Math.Sqrt((174.49694289101447 / (299)) + (5.600803674158737 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17633127876409468 = (387.10644013030446 - 318.8474665243114) / 387.10644013030446 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 9, Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 98.35822443991403 < 123.8398407153352.
IsChangePoint: Marked as a change because one of 5/1/2023 3:42:23 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 45.48066084093572 (T) = (0 -96.96401407569631) / Math.Sqrt((50.25309043635497 / (299)) + (1.0242495275442351 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18899988691555858 = (119.56103644291404 - 96.96401407569631) / 119.56103644291404 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 0, Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 95.03937817628926 < 105.99050854481995.
IsChangePoint: Marked as a change because one of 5/19/2023 8:24:48 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 8.532392156661887 (T) = (0 -90.77127644223991) / Math.Sqrt((28.362644717415495 / (299)) + (76.72354358198385 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.19995713699911993 = (113.45801661396742 - 90.77127644223991) / 113.45801661396742 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 0, Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 59.85848992335671 < 82.33675603170195.
IsChangePoint: Marked as a change because one of 3/10/2023 12:53:36 AM, 4/7/2023 3:57:10 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 11.72724068031488 (T) = (0 -64.98628694014475) / Math.Sqrt((23.649812376947175 / (299)) + (38.51056233434291 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23937754375003725 = (85.43829649803077 - 64.98628694014475) / 85.43829649803077 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 12, Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 137.84030499617066 < 148.860366576317.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 23.77736809219538 (T) = (0 -126.29050479533034) / Math.Sqrt((5.619142690778876 / (299)) + (22.96497131575985 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20100110164119614 = (158.060924808206 - 126.29050479533034) / 158.060924808206 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 4, Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 98.32640177697043 < 103.86595915327725.
IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 3/25/2023 12:18:49 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 33.01569107701754 (T) = (0 -98.51019923417678) / Math.Sqrt((48.91820352784286 / (299)) + (0.053030425455281964 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12067959308778292 = (112.02992499639679 - 98.51019923417678) / 112.02992499639679 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 4, Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 137.38424136975505 < 169.36385397490832.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 19.947389219968215 (T) = (0 -143.95961375886188) / Math.Sqrt((27.773340910069866 / (299)) + (46.88250685535872 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21042398644564939 = (182.3252116168197 - 143.95961375886188) / 182.3252116168197 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 4, Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 282.91788462242226 < 319.701575463703.
IsChangePoint: Marked as a change because one of 3/5/2023 4:40:20 PM, 3/6/2023 6:03:21 AM, 4/27/2023 1:43:27 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 14.539614738717646 (T) = (0 -284.84880360620184) / Math.Sqrt((206.68841148860398 / (299)) + (182.591800157393 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1638429576493265 = (340.6642403027683 - 284.84880360620184) / 340.6642403027683 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 8, Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 110.23721948706205 < 112.65635271919972.
IsChangePoint: Marked as a change because one of 3/6/2023 11:36:59 AM, 4/12/2023 8:39:35 PM, 5/11/2023 9:33:40 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 16.488774675209484 (T) = (0 -110.52335729444191) / Math.Sqrt((16.031311889985265 / (299)) + (2.7163057291447656 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07101647444050425 = (118.97235446438879 - 110.52335729444191) / 118.97235446438879 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 7, Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 77.70776077398016 < 91.709664967096.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 21.295203799383618 (T) = (0 -78.72746646323831) / Math.Sqrt((5.983830154770791 / (299)) + (10.794603513831378 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19963871958376478 = (98.36491143386569 - 78.72746646323831) / 98.36491143386569 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 2, Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 215.53186577797774 < 258.44019845162194.
IsChangePoint: Marked as a change because one of 4/8/2023 3:09:32 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 38.822377804421194 (T) = (0 -218.89917268376772) / Math.Sqrt((54.22608749845844 / (299)) + (19.72058843133338 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18773538513215324 = (269.4924396274261 - 218.89917268376772) / 269.4924396274261 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 13, Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 86.7325164048232 < 115.65595921566926.
IsChangePoint: Marked as a change because one of 4/20/2023 12:14:19 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 65.05634067897456 (T) = (0 -86.64255401512513) / Math.Sqrt((38.66624401851387 / (299)) + (0.18805412917661532 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22161871015545506 = (111.31119818210047 - 86.64255401512513) / 111.31119818210047 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 13, Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 40.252108874681994 < 42.48546944600725.
IsChangePoint: Marked as a change because one of 4/23/2023 10:00:36 AM, 5/5/2023 3:18:41 PM, 5/9/2023 1:22:51 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 26.87988884508081 (T) = (0 -40.3838275353305) / Math.Sqrt((6.697363732850745 / (299)) + (0.3495502665168746 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12874606982878897 = (46.35138636034001 - 40.3838275353305) / 46.35138636034001 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 10, Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 90.97339410719303 < 104.41940927242464.
IsChangePoint: Marked as a change because one of 4/4/2023 4:23:54 PM, 4/6/2023 9:09:21 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 39.3198013821324 (T) = (0 -90.00739705706086) / Math.Sqrt((8.353787581028305 / (299)) + (3.295417992717997 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18814719546717515 = (110.86664547381221 - 90.00739705706086) / 110.86664547381221 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 5, Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 136.61382711082365 < 168.08760013182058.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 34.03738138274456 (T) = (0 -137.57282844549312) / Math.Sqrt((9.608808098015322 / (299)) + (16.681983144930317 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22103645490048443 = (176.61009852254082 - 137.57282844549312) / 176.61009852254082 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 10, Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 95.20158034736347 < 120.8139962294406.
IsChangePoint: Marked as a change because one of 5/2/2023 9:04:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 43.14254401510209 (T) = (0 -96.7631381417477) / Math.Sqrt((47.19646496995238 / (299)) + (0.9181388458945973 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17567583130850342 = (117.3848126949209 - 96.7631381417477) / 117.3848126949209 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 3, Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 171.2881366352989 < 197.82874602533022.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/19/2023 1:39:34 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 29.06979202232637 (T) = (0 -171.26575558272145) / Math.Sqrt((29.693134163524928 / (299)) + (16.08550339145953 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16404527884155307 = (204.87444026320622 - 171.26575558272145) / 204.87444026320622 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 12, Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 41.08319378085286 < 42.413273407395536.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/17/2023 10:06:19 AM, 5/5/2023 3:18:41 PM, 5/9/2023 1:22:51 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 27.158588278984915 (T) = (0 -40.42930134516874) / Math.Sqrt((10.34190239189536 / (299)) + (0.17283571187807115 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12815679489256954 = (46.37221590800487 - 40.42930134516874) / 46.37221590800487 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedDictionaryDeepCopy - Duration of single invocation 10.75 μs 6.38 μs 0.59 0.11 True
ImmutableHashSet - Duration of single invocation 108.74 μs 58.19 μs 0.54 0.01 True
ConcurrentBag - Duration of single invocation 6.28 μs 5.54 μs 0.88 0.04 False
SortedSet - Duration of single invocation 9.76 μs 8.86 μs 0.91 0.11 False
FrozenDictionaryOptimized - Duration of single invocation 73.94 μs 55.73 μs 0.75 0.30 False
HashSet - Duration of single invocation 6.73 μs 5.71 μs 0.85 0.08 False
LinkedList - Duration of single invocation 7.58 μs 5.49 μs 0.72 0.10 False
ImmutableSortedDictionary - Duration of single invocation 70.35 μs 45.48 μs 0.65 0.04 True
SortedDictionary - Duration of single invocation 55.16 μs 32.63 μs 0.59 0.04 True
Queue - Duration of single invocation 112.42 ns 103.76 ns 0.92 0.09 False
ImmutableSortedSet - Duration of single invocation 14.76 μs 12.18 μs 0.82 0.02 False
ConcurrentQueue - Duration of single invocation 4.90 μs 4.01 μs 0.82 0.04 False
SortedList - Duration of single invocation 7.66 μs 6.45 μs 0.84 0.03 False
List - Duration of single invocation 111.55 ns 100.13 ns 0.90 0.04 False
ImmutableDictionary - Duration of single invocation 117.36 μs 80.17 μs 0.68 0.01 True
ConcurrentDictionary - Duration of single invocation 19.92 μs 16.86 μs 0.85 0.01 True
ImmutableArray - Duration of single invocation 110.83 ns 101.01 ns 0.91 0.10 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<Int32>.SortedDictionaryDeepCopy(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.3822234202174455 < 9.916605273299966.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 210.13722640720783 (T) = (0 -6366.841307117249) / Math.Sqrt((55445.05248089621 / (299)) + (2512.466103414606 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.39109215216684495 = (10456.1656246904 - 6366.841307117249) / 10456.1656246904 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.ImmutableHashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 58.19039981617647 < 103.64354892828526.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 148.63816433558642 (T) = (0 -58641.719700175236) / Math.Sqrt((2035773.0842848197 / (299)) + (1536161.7748478074 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4725893455973566 = (111187.96939472924 - 58641.719700175236) / 111187.96939472924 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.ConcurrentBag(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.535228545906007 < 5.979459041649304.
IsChangePoint: Marked as a change because one of 3/17/2023 9:25:32 PM, 3/20/2023 8:31:51 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 32.93275035499266 (T) = (0 -5638.971805139249) / Math.Sqrt((18456.998519017507 / (299)) + (4815.313704307936 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10826210103640468 = (6323.575359635418 - 5638.971805139249) / 6323.575359635418 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.SortedSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.857662346823604 < 9.012774422856143.
IsChangePoint: Marked as a change because one of 3/16/2023 7:54:34 AM, 3/16/2023 8:14:39 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 9.922917572045298 (T) = (0 -8705.068066059528) / Math.Sqrt((152833.80723858273 / (299)) + (22369.414037389175 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.05110017828220378 = (9173.85362166126 - 8705.068066059528) / 9173.85362166126 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.FrozenDictionaryOptimized(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 55.732262241054606 < 66.72378251499019.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 26.57009086821847 (T) = (0 -55265.89528217555) / Math.Sqrt((10471835.806976734 / (265)) + (4893198.2957599675 / (13))) is greater than 1.968596344330496 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (265) + (13) - 2, .975) and 0.2366732797593457 = (72401.3634224043 - 55265.89528217555) / 72401.3634224043 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.HashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.714947761502813 < 6.18372915501342.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 10.106797228466803 (T) = (0 -5458.5953494480555) / Math.Sqrt((215077.6553987726 / (299)) + (334582.0325300594 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2314553204164371 = (7102.50879936584 - 5458.5953494480555) / 7102.50879936584 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.LinkedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.491001334245818 < 7.207957333478118.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 60.516530875948085 (T) = (0 -5538.377668865714) / Math.Sqrt((103472.68593420749 / (299)) + (10432.38835082581 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2702388969272821 = (7589.30236970144 - 5538.377668865714) / 7589.30236970144 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.ImmutableSortedDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 45.48036989795918 < 66.52994072956191.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 109.24788175907109 (T) = (0 -45360.2288367179) / Math.Sqrt((1149432.4171870484 / (299)) + (580875.5908743032 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3466417452720195 = (69426.27342422298 - 45360.2288367179) / 69426.27342422298 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 32.631545546558705 < 50.91603665338433.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 132.11703101839103 (T) = (0 -32469.88044875342) / Math.Sqrt((955458.855211813 / (299)) + (293536.0883404966 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.39513071193228877 = (53680.82177304831 - 32469.88044875342) / 53680.82177304831 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.Queue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 103.75543178406885 < 108.42456633840705.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/11/2023 9:15:05 PM, 5/16/2023 5:40:18 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.084899128275888 (T) = (0 -104.99261630769142) / Math.Sqrt((15.26449725739584 / (299)) + (5.777360007703447 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.057616774883522674 = (111.41180520771098 - 104.99261630769142) / 111.41180520771098 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.ImmutableSortedSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.17719955969956 < 14.053909952910738.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 48.78543492905794 (T) = (0 -11978.628496525102) / Math.Sqrt((138030.25870990212 / (299)) + (25368.166425581134 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1667098724881566 = (14375.099501409672 - 11978.628496525102) / 14375.099501409672 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.ConcurrentQueue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.0084193336234355 < 4.7001030702788205.
IsChangePoint: Marked as a change because one of 3/31/2023 8:05:26 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 108.13076260707629 (T) = (0 -3989.3926200973856) / Math.Sqrt((11541.97971550917 / (299)) + (537.9073281782629 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1951058883398093 = (4956.419188940002 - 3989.3926200973856) / 4956.419188940002 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.SortedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.447565102630647 < 7.295426147155597.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 71.76651051090401 (T) = (0 -6434.60099209557) / Math.Sqrt((59588.82603677329 / (299)) + (1072.5129210393814 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1577005342150196 = (7639.32692999971 - 6434.60099209557) / 7639.32692999971 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.List(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 100.12798868723772 < 103.43816611553653.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/16/2023 5:40:18 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 11.247648131863748 (T) = (0 -102.9158938544838) / Math.Sqrt((9.245643971140984 / (299)) + (5.867405555565792 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.05424238929604396 = (108.81846753300815 - 102.9158938544838) / 108.81846753300815 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.ImmutableDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 80.16775446775446 < 111.42531078296706.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 134.43576918586044 (T) = (0 -79798.40851786363) / Math.Sqrt((872722.4747249499 / (299)) + (972057.8875417378 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3195337371925545 = (117270.19086682411 - 79798.40851786363) / 117270.19086682411 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.86099617117117 < 18.905377508861967.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 83.85639786156348 (T) = (0 -16897.625867154864) / Math.Sqrt((51175.63861908014 / (299)) + (16292.223241775633 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15774961611298796 = (20062.473333846152 - 16897.625867154864) / 20062.473333846152 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.ImmutableArray(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 101.00661168252603 < 108.260460393737.
IsChangePoint: Marked as a change because one of 4/19/2023 3:30:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 39.8287702536611 (T) = (0 -100.86454208460646) / Math.Sqrt((12.83609346064026 / (299)) + (1.6037373779195951 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13869320446333527 = (117.10640460204378 - 100.86454208460646) / 117.10640460204378 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in GuardedDevirtualization.TwoClassInterface

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Call - Duration of single invocation 6.44 ns 1.18 ns 0.18 0.04 True
Call - Duration of single invocation 4.95 ns 0.68 ns 0.14 0.03 True
Call - Duration of single invocation 2.24 ns 0.49 ns 0.22 0.09 True
Call - Duration of single invocation 6.66 ns 1.24 ns 0.19 0.04 True
Call - Duration of single invocation 4.42 ns 0.82 ns 0.19 0.02 True
Call - Duration of single invocation 1.96 ns 0.49 ns 0.25 0.13 True
Call - Duration of single invocation 7.62 ns 1.50 ns 0.20 0.01 True
Call - Duration of single invocation 7.88 ns 1.75 ns 0.22 0.03 True
Call - Duration of single invocation 7.33 ns 1.23 ns 0.17 0.02 True
Call - Duration of single invocation 7.77 ns 1.87 ns 0.24 0.04 True
Call - Duration of single invocation 8.01 ns 2.14 ns 0.27 0.02 True

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'GuardedDevirtualization.TwoClassInterface*'

Payloads

Baseline
Compare

Histogram

GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.80)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.179636563180001 < 6.119650143669277.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 34.76270997714384 (T) = (0 -1.2533467770950901) / Math.Sqrt((0.023882951109872118 / (299)) + (0.2725376597361558 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.8009377067065523 = (6.2962540838785 - 1.2533467770950901) / 6.2962540838785 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.6758659499985625 < 4.696643240198315.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 142.18096688115568 (T) = (0 -0.7441970206903504) / Math.Sqrt((0.0040581414020921455 / (299)) + (0.010913701785241163 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.8480291319729826 = (4.896971573249466 - 0.7441970206903504) / 4.896971573249466 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 1.00)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.4948063685695238 < 1.8603668078581521.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 209.9036621417827 (T) = (0 -0.4946389355698123) / Math.Sqrt((0.015875480575778263 / (299)) + (5.123214887284659E-06 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7563113693983121 = (2.029798987127577 - 0.4946389355698123) / 2.029798987127577 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.20)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.242049298679868 < 6.324499080525603.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 179.7541620466561 (T) = (0 -1.2120251629666223) / Math.Sqrt((0.027054351225671347 / (299)) + (0.010130776861736914 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.813915931410062 = (6.513320415610042 - 1.2120251629666223) / 6.513320415610042 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.90)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.8237797338788172 < 4.203239287707631.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 141.3023745552807 (T) = (0 -0.6894077664154048) / Math.Sqrt((0.003585981528073806 / (299)) + (0.008545186365314305 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.8413352574114727 = (4.345059621741415 - 0.6894077664154048) / 4.345059621741415 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.00)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.490759701790938 < 2.1214443810598675.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 196.78293043959795 (T) = (0 -0.4918604985038856) / Math.Sqrt((0.019775673820872985 / (299)) + (9.388253817618303E-07 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7650078355357313 = (2.093093187278057 - 0.4918604985038856) / 2.093093187278057 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.30)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.501079678152094 < 7.246668322719402.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 93.6751119108988 (T) = (0 -1.636692520381108) / Math.Sqrt((0.024986296504047822 / (299)) + (0.049803258572932475 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.781705371800855 = (7.497630765737359 - 1.636692520381108) / 7.497630765737359 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.60)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.7547284889334522 < 7.489195197251454.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 82.02185143779295 (T) = (0 -1.9670219476940463) / Math.Sqrt((0.027773841287552836 / (299)) + (0.06269425391223866 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.745126477000918 = (7.717639417967832 - 1.9670219476940463) / 7.717639417967832 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.70)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2334543164280636 < 6.944505542924206.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 47.130452414170875 (T) = (0 -1.4537317527495106) / Math.Sqrt((0.016888183656029957 / (299)) + (0.19188895453325605 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7978323500422941 = (7.1907239019380045 - 1.4537317527495106) / 7.1907239019380045 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.40)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.8708769017145621 < 7.385901324684427.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 129.20114224005252 (T) = (0 -2.0250319152843543) / Math.Sqrt((0.030174437583083635 / (299)) + (0.023157561654432817 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7346113535397276 = (7.630439140083914 - 2.0250319152843543) / 7.630439140083914 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.50)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.1429537766382656 < 7.608327324927535.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 206.48572306367197 (T) = (0 -2.188260464737427) / Math.Sqrt((0.0026844845256782973 / (299)) + (0.010338426896634922 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7279643728796701 = (8.044021615483073 - 2.188260464737427) / 8.044021615483073 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Random

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Next_int - Duration of single invocation 6.11 ns 4.14 ns 0.68 0.05 True
Next_int_int - Duration of single invocation 7.73 ns 4.30 ns 0.56 0.07 True
Next_int_unseeded - Duration of single invocation 4.34 ns 2.16 ns 0.50 0.07 True
NextDouble - Duration of single invocation 4.75 ns 3.54 ns 0.75 0.02 True
Next_long - Duration of single invocation 31.08 ns 28.84 ns 0.93 0.05 False
Next_int_int_unseeded - Duration of single invocation 3.84 ns 2.20 ns 0.57 0.04 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Random.Next_int

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.13504080946309 < 5.795541816412231.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 90.05437434414111 (T) = (0 -4.145077023759226) / Math.Sqrt((0.13630942445532812 / (299)) + (0.0005077899695803597 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.325844713936498 = (6.1485493171209535 - 4.145077023759226) / 6.1485493171209535 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Random.Next_int_int

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.296963358047787 < 6.160262687730303.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 152.02167809262434 (T) = (0 -4.293626041480182) / Math.Sqrt((0.08137943432025317 / (299)) + (7.345127058735816E-05 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.37112974807044696 = (6.827522892530079 - 4.293626041480182) / 6.827522892530079 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Random.Next_int_unseeded

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.1624188920393936 < 4.123600404622243.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 162.47268630463896 (T) = (0 -2.171715088263299) / Math.Sqrt((0.05303236697353346 / (299)) + (0.00013339571265783711 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5061160297295706 = (4.397217198756547 - 2.171715088263299) / 4.397217198756547 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Random.NextDouble

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.5426026554340653 < 4.515098169490568.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 137.70092359620622 (T) = (0 -3.554182789823997) / Math.Sqrt((0.02064423877591661 / (299)) + (0.00011755297442236962 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2550446365006583 = (4.771001007535047 - 3.554182789823997) / 4.771001007535047 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Random.Next_long

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 28.83870797574623 < 29.550555511514812.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.129440371565583 (T) = (0 -29.027797384853336) / Math.Sqrt((0.2962384130855362 / (299)) + (0.20855757936345362 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.051715333292822435 = (30.61084756927413 - 29.027797384853336) / 30.61084756927413 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Random.Next_int_int_unseeded

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.2009980677410557 < 3.6479423396991923.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 111.96367705491218 (T) = (0 -2.205845718427303) / Math.Sqrt((0.04352132319949487 / (299)) + (0.00026787967143980485 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.395510022602089 = (3.649102219895509 - 2.205845718427303) / 3.649102219895509 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 32.29 ns 24.97 ns 0.77 0.03 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 24.967525924881144 < 30.78834487379806.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 49.6676506021262 (T) = (0 -24.559126451068398) / Math.Sqrt((0.724049258902581 / (299)) + (0.2684631325628147 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23500097347820764 = (32.10347412170045 - 24.559126451068398) / 32.10347412170045 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 70.15 μs 65.23 μs 0.93 0.02 False
Count - Duration of single invocation 109.05 μs 91.61 μs 0.84 0.26 False
Count - Duration of single invocation 6.41 ms 4.51 ms 0.70 0.11 False
Count - Duration of single invocation 1.27 ms 1.15 ms 0.90 0.10 False
Count - Duration of single invocation 2.87 ms 2.47 ms 0.86 0.03 True
Count - Duration of single invocation 40.42 ms 32.07 ms 0.79 0.24 False
Count - Duration of single invocation 2.58 ms 2.01 ms 0.78 0.01 False
Count - Duration of single invocation 184.63 μs 153.21 μs 0.83 0.08 False
Count - Duration of single invocation 406.92 μs 374.63 μs 0.92 0.06 False
Count - Duration of single invocation 43.69 ms 34.36 ms 0.79 0.23 False
Count - Duration of single invocation 270.11 μs 233.93 μs 0.87 0.01 False
Count - Duration of single invocation 364.46 μs 294.70 μs 0.81 0.13 False
Count - Duration of single invocation 274.73 μs 205.90 μs 0.75 0.06 True
Count - Duration of single invocation 3.73 ms 3.51 ms 0.94 0.02 False
Count - Duration of single invocation 16.69 ms 13.37 ms 0.80 0.24 False
Count - Duration of single invocation 2.64 ms 2.19 ms 0.83 0.16 False
Count - Duration of single invocation 56.39 μs 42.09 μs 0.75 0.05 False
Count - Duration of single invocation 56.80 μs 48.49 μs 0.85 0.11 False
Count - Duration of single invocation 19.66 ms 12.72 ms 0.65 0.44 False
Count - Duration of single invocation 130.14 μs 119.40 μs 0.92 0.04 False
Count - Duration of single invocation 75.91 μs 69.61 μs 0.92 0.01 False
Count - Duration of single invocation 11.33 ms 9.08 ms 0.80 0.41 False
Count - Duration of single invocation 2.48 ms 2.25 ms 0.91 0.08 False
Count - Duration of single invocation 249.29 μs 196.53 μs 0.79 0.08 False
Count - Duration of single invocation 2.37 ms 2.06 ms 0.87 0.14 False
Count - Duration of single invocation 97.92 μs 81.58 μs 0.83 0.21 False
Count - Duration of single invocation 11.49 ms 9.81 ms 0.85 0.12 False
Count - Duration of single invocation 20.55 ms 17.22 ms 0.84 0.30 False
Count - Duration of single invocation 32.59 ms 26.70 ms 0.82 0.18 False
Count - Duration of single invocation 17.85 ms 11.81 ms 0.66 0.17 False
Count - Duration of single invocation 1.70 ms 1.57 ms 0.93 0.12 False
Count - Duration of single invocation 41.98 μs 23.32 μs 0.56 0.04 False
Count - Duration of single invocation 33.66 ms 25.36 ms 0.75 0.04 True
Count - Duration of single invocation 47.82 μs 29.22 μs 0.61 0.06 False
Count - Duration of single invocation 416.24 μs 320.48 μs 0.77 0.02 True
Count - Duration of single invocation 57.78 μs 36.71 μs 0.64 0.41 False
Count - Duration of single invocation 1.13 ms 1.03 ms 0.91 0.01 False
Count - Duration of single invocation 3.78 ms 3.56 ms 0.94 0.00 False
Count - Duration of single invocation 140.22 μs 109.30 μs 0.78 0.09 False
Count - Duration of single invocation 9.96 ms 7.17 ms 0.72 0.30 False
Count - Duration of single invocation 33.56 ms 23.61 ms 0.70 0.15 True
Count - Duration of single invocation 70.89 ns 59.43 ns 0.84 0.03 False
Count - Duration of single invocation 6.96 ms 6.32 ms 0.91 0.09 False
Count - Duration of single invocation 56.49 μs 52.08 μs 0.92 0.06 False
Count - Duration of single invocation 52.98 μs 44.06 μs 0.83 0.04 False
Count - Duration of single invocation 1.18 ms 911.70 μs 0.77 0.01 False
Count - Duration of single invocation 979.64 μs 843.62 μs 0.86 0.18 False
Count - Duration of single invocation 165.15 μs 129.45 μs 0.78 0.15 False
Count - Duration of single invocation 41.90 μs 23.20 μs 0.55 0.04 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 65.23424509660423 < 66.73092165493406.
IsChangePoint: Marked as a change because one of 4/23/2023 2:59:55 PM, 5/1/2023 10:17:01 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 28.879748137252804 (T) = (0 -65116.9014787637) / Math.Sqrt((1705678.1920055328 / (299)) + (451430.35114030243 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08187537350534023 = (70923.81535105512 - 65116.9014787637) / 70923.81535105512 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 91.60610684697855 < 102.92197468171295.
IsChangePoint: Marked as a change because one of 3/28/2023 3:45:48 AM, 4/25/2023 1:57:48 PM, 5/5/2023 3:18:41 PM, 5/16/2023 5:40:18 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 11.823899631346015 (T) = (0 -90223.23521365131) / Math.Sqrt((92347344.06378064 / (299)) + (3579587.228102694 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.08146523539783018 = (98225.17197019565 - 90223.23521365131) / 98225.17197019565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sher[a-z]+|Hol[a-z]+", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.505525194805195 < 6.062288427350428.
IsChangePoint: Marked as a change because one of 3/28/2023 3:45:48 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 50.31496383721546 (T) = (0 -4629813.270158488) / Math.Sqrt((63583221978.398834 / (299)) + (10614569994.043036 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2585112052578477 = (6243942.3265573075 - 4629813.270158488) / 6243942.3265573075 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the\s+\w+", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1512260493827162 < 1.2502238744646006.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 14.935434196609446 (T) = (0 -1101533.4065135228) / Math.Sqrt((1600781698.251554 / (299)) + (2604132441.4339137 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1627944236325437 = (1315726.3133541897 - 1101533.4065135228) / 1315726.3133541897 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\s[a-zA-Z]{0,12}ing\s", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.467369623233909 < 2.7196692920678456.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 11.178219540386488 (T) = (0 -2581347.773067912) / Math.Sqrt((763942552.7764708 / (299)) + (7696857055.210068 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09551022527196688 = (2853926.9820314837 - 2581347.773067912) / 2853926.9820314837 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{Ll}", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 32.06655875 < 38.563287458333335.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 22.756396556050138 (T) = (0 -32467577.355240908) / Math.Sqrt((2986026621943.75 / (299)) + (1956165165464.2517 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21921479843392172 = (41583238.62967472 - 32467577.355240908) / 41583238.62967472 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.0139529776674934 < 2.4123528226495723.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/15/2023 9:18:21 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 38.93809654940303 (T) = (0 -2010356.5844588592) / Math.Sqrt((20434025099.783897 / (299)) + (354450064.6415312 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.14778182805389797 = (2358969.3937974405 - 2010356.5844588592) / 2358969.3937974405 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 153.20587826797384 < 174.47114643182854.
IsChangePoint: Marked as a change because one of 4/10/2023 4:31:44 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 34.18947519201484 (T) = (0 -155099.07980415082) / Math.Sqrt((22092777.113936286 / (299)) + (8992684.102648696 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1616947605884002 = (185015.04286554825 - 155099.07980415082) / 185015.04286554825 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 374.63110119047616 < 386.45322842261896.
IsChangePoint: Marked as a change because one of 3/16/2023 2:58:09 AM, 3/17/2023 8:17:18 AM, 5/9/2023 1:22:51 PM, 5/11/2023 9:33:40 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 42.06411079916813 (T) = (0 -374052.73147936875) / Math.Sqrt((190911600.25828746 / (299)) + (2524106.3441306083 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09306560828302601 = (412436.37345279875 - 374052.73147936875) / 412436.37345279875 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{L}", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 34.356627857142854 < 41.67855946428571.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 22.202223748188942 (T) = (0 -33535503.44639335) / Math.Sqrt((2938585443587.602 / (299)) + (2391044923795.42 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22565788396199277 = (43308381.07835441 - 33535503.44639335) / 43308381.07835441 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 233.93472636815915 < 257.8742282815066.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/15/2023 9:18:21 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 43.50577429385159 (T) = (0 -233732.70255438855) / Math.Sqrt((95853222.64209133 / (299)) + (540165.8234634936 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.0976482033995575 = (259026.1397328213 - 233732.70255438855) / 259026.1397328213 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 294.69617138364777 < 347.5597185077519.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 75.9084959631905 (T) = (0 -288547.479436095) / Math.Sqrt((232825374.02679598 / (299)) + (5252978.035774627 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22245656734023303 = (371101.4295999539 - 288547.479436095) / 371101.4295999539 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 205.89729989035087 < 259.3245104418886.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 56.36189648837257 (T) = (0 -207249.9233746695) / Math.Sqrt((39823345.60692167 / (299)) + (12322218.375804788 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22042952841410884 = (265851.42835523013 - 207249.9233746695) / 265851.42835523013 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+\s+Holmes", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.5117379275653917 < 3.5448159111607143.
IsChangePoint: Marked as a change because one of 3/28/2023 8:13:28 AM, 4/10/2023 1:33:42 PM, 5/15/2023 6:46:12 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 33.70813983903193 (T) = (0 -3533726.8151475852) / Math.Sqrt((12697586609.396057 / (299)) + (505865654.9020105 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06875690354500573 = (3794634.1063891747 - 3533726.8151475852) / 3794634.1063891747 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.370920833333335 < 15.6481518125.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 35.029426572200016 (T) = (0 -13700353.993999423) / Math.Sqrt((441141897281.3549 / (299)) + (82523084575.56349 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18443917060217738 = (16798690.54539465 - 13700353.993999423) / 16798690.54539465 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.187782 < 2.5089107983630954.
IsChangePoint: Marked as a change because one of 4/4/2023 4:23:54 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 20.43501559323555 (T) = (0 -2134145.8205128205) / Math.Sqrt((23080161653.421066 / (299)) + (4879797143.703699 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16922757425131482 = (2568869.3490152205 - 2134145.8205128205) / 2568869.3490152205 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 42.09049191374664 < 53.554983475719425.
IsChangePoint: Marked as a change because one of 3/10/2023 12:53:36 AM, 3/28/2023 3:45:48 AM, 4/7/2023 3:57:10 PM, 4/25/2023 6:46:56 PM, 5/9/2023 1:22:51 PM, 5/16/2023 5:40:18 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 17.36814575460689 (T) = (0 -42372.71281031566) / Math.Sqrt((43000489.794175364 / (299)) + (55988.95039145974 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.13543244146222433 = (49010.29698821883 - 42372.71281031566) / 49010.29698821883 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 48.48783889853953 < 53.9236249342848.
IsChangePoint: Marked as a change because one of 4/26/2023 12:47:57 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 41.163525685294964 (T) = (0 -48468.05657850004) / Math.Sqrt((2544067.5634927573 / (299)) + (537333.5011741163 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15938626116813776 = (57657.940073466394 - 48468.05657850004) / 57657.940073466394 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-zA-Z]+ing", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.71734642857143 < 17.404240488945575.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 22.750898220831644 (T) = (0 -13497303.590448013) / Math.Sqrt((983381167854.375 / (299)) + (583172385904.3848 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27000053168818716 = (18489470.43983703 - 13497303.590448013) / 18489470.43983703 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 119.39651303088802 < 122.98878002802824.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 18.310152371551798 (T) = (0 -111750.15893377116) / Math.Sqrt((4426170.537771035 / (299)) + (12589790.68370437 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13976338749197345 = (129906.30404344533 - 111750.15893377116) / 129906.30404344533 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 69.61338690476191 < 72.04524733293074.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/13/2023 2:14:21 PM, 3/28/2023 8:13:28 AM, 4/8/2023 3:09:32 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 28.719883890007168 (T) = (0 -70122.78214900586) / Math.Sqrt((1821122.3357167377 / (299)) + (523510.4873522509 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08103930250209815 = (76306.61718170592 - 70122.78214900586) / 76306.61718170592 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.075707160493826 < 11.976611067821068.
IsChangePoint: Marked as a change because one of 5/2/2023 10:35:24 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 15.24049915309335 (T) = (0 -9063602.840318684) / Math.Sqrt((1239457355001.6804 / (299)) + (197249905996.1896 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18943912298571844 = (11181890.339569138 - 9063602.840318684) / 11181890.339569138 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+\s+Holmes\s+\w+", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.2478713043478256 < 2.3606194074728903.
IsChangePoint: Marked as a change because one of 4/18/2023 7:42:00 PM, 4/19/2023 8:38:27 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 11.863066428871955 (T) = (0 -2266414.5843441878) / Math.Sqrt((2372236349.963666 / (299)) + (4258082246.2545495 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08748435492813447 = (2483699.426507581 - 2266414.5843441878) / 2483699.426507581 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 196.5290745192308 < 234.85369645182288.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/25/2023 10:49:06 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 83.19657239002066 (T) = (0 -197469.52494775708) / Math.Sqrt((52543498.04912078 / (299)) + (2223940.199851696 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19878950738306733 = (246463.9776530852 - 197469.52494775708) / 246463.9776530852 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{Lu}", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.0572921763085397 < 2.305597789528433.
IsChangePoint: Marked as a change because one of 3/28/2023 3:45:48 AM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 10.588024739847897 (T) = (0 -1998803.5378882263) / Math.Sqrt((22332864122.01667 / (299)) + (15575486191.787231 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1589460777103459 = (2376546.2414666084 - 1998803.5378882263) / 2376546.2414666084 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 81.58023316062176 < 93.6512560546875.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 41.005474355788024 (T) = (0 -81483.4761231586) / Math.Sqrt((19661578.86479282 / (299)) + (2257428.633944283 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1975795265270826 = (101547.10506138246 - 81483.4761231586) / 101547.10506138246 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\b\w+n\b", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.813876923076924 < 10.821760765151515.
IsChangePoint: Marked as a change because one of 3/6/2023 11:36:59 AM, 3/17/2023 10:06:19 AM, 3/28/2023 8:13:28 AM, 5/1/2023 3:42:23 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 41.003077158404665 (T) = (0 -9700334.50024926) / Math.Sqrt((956492539072.1519 / (299)) + (4683864378.357251 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2013925870743992 = (12146562.056960212 - 9700334.50024926) / 12146562.056960212 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\s[a-zA-Z]{0,12}ing\s", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.218510714285713 < 18.765904430555555.
IsChangePoint: Marked as a change because one of 3/25/2023 10:49:06 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 10.977399261318316 (T) = (0 -15348533.665821357) / Math.Sqrt((579989576351.4326 / (299)) + (2105814132811.1765 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22454860667653007 = (19793031.256336793 - 15348533.665821357) / 19793031.256336793 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{L}", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 26.69606 < 30.916817416666664.
IsChangePoint: Marked as a change because one of 3/14/2023 5:06:59 PM, 4/18/2023 7:42:00 PM, 5/5/2023 3:18:41 PM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 13.91790405235091 (T) = (0 -27768446.50641025) / Math.Sqrt((4116866051717.2056 / (299)) + (1598243671580.8975 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.12086518994882846 = (31586107.373900868 - 27768446.50641025) / 31586107.373900868 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{Ll}", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.80853319838057 < 16.383251969887954.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/23/2023 2:59:55 PM, 5/5/2023 9:43:39 PM, 5/15/2023 9:18:21 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 10.777709247579685 (T) = (0 -12995241.1315161) / Math.Sqrt((2204309152194.9385 / (299)) + (1255913779407.8506 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.15542534179566272 = (15386728.698615557 - 12995241.1315161) / 15386728.698615557 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: ".*", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.5740830801687762 < 1.6113419914641205.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.623853033979593 (T) = (0 -1476019.323648937) / Math.Sqrt((14773084783.656912 / (299)) + (5621616158.074329 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15806325499519663 = (1753123.773734945 - 1476019.323648937) / 1753123.773734945 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.317607898009946 < 39.91137542002687.
IsChangePoint: Marked as a change because one of 3/10/2023 12:53:36 AM, 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 5/2/2023 3:18:41 PM, 5/5/2023 3:18:41 PM, 5/9/2023 1:22:51 PM, 5/11/2023 9:15:05 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 32.70389579155251 (T) = (0 -23289.439968013783) / Math.Sqrt((59366409.400096804 / (299)) + (208.02865971558597 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3848948530094896 = (37862.5346933946 - 23289.439968013783) / 37862.5346933946 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 25.35618666666667 < 32.52165976785714.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 137.3351073313962 (T) = (0 -25422597.49718231) / Math.Sqrt((468849458420.03973 / (299)) + (30938661868.32535 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25341269718518655 = (34051741.03729465 - 25422597.49718231) / 34051741.03729465 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\s+Holmes", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 29.216293806848583 < 45.41224668705403.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 4/8/2023 3:09:32 AM, 4/17/2023 6:40:18 PM, 5/1/2023 3:42:23 PM, 5/5/2023 3:18:41 PM, 5/9/2023 1:22:51 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 16.9539990579427 (T) = (0 -29326.127632827647) / Math.Sqrt((73314312.56414911 / (299)) + (18934.95282829068 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22307121165308047 = (37746.223428308265 - 29326.127632827647) / 37746.223428308265 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 320.484906462585 < 401.8885085758836.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 37.85790269038059 (T) = (0 -327645.9918646841) / Math.Sqrt((21723104.487136163 / (299)) + (74780122.67808984 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2180584086569944 = (419015.9412059708 - 327645.9918646841) / 419015.9412059708 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 36.70500083836352 < 54.958670743217844.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/11/2023 10:53:42 PM, 4/25/2023 3:46:39 AM, 5/19/2023 12:51:11 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 26.899072902028575 (T) = (0 -37093.84650112889) / Math.Sqrt((73382912.14483587 / (299)) + (81417.13963318997 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.26643260432991006 = (50566.378386058 - 37093.84650112889) / 50566.378386058 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sher[a-z]+|Hol[a-z]+", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0313804979253114 < 1.0763396546003017.
IsChangePoint: Marked as a change because one of 3/7/2023 4:52:41 AM, 3/7/2023 4:31:45 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 45.82697811198733 (T) = (0 -1029141.4452916338) / Math.Sqrt((350770749.38324493 / (299)) + (63661986.08348637 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09886412458039844 = (1142049.1330593494 - 1029141.4452916338) / 1142049.1330593494 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+\s+Holmes\s+\w+", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.555496 < 3.571027754166666.
IsChangePoint: Marked as a change because one of 3/28/2023 8:13:28 AM, 4/10/2023 1:33:42 PM, 5/15/2023 9:18:21 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 32.40817135600814 (T) = (0 -3550375.7824035943) / Math.Sqrt((13325979845.64848 / (299)) + (119597234.18813612 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.05980214317263902 = (3776200.675870626 - 3550375.7824035943) / 3776200.675870626 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 109.29867766203705 < 132.41112072666456.
IsChangePoint: Marked as a change because one of 4/11/2023 5:52:15 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 34.31603903310394 (T) = (0 -111239.62807081969) / Math.Sqrt((15866769.165451499 / (299)) + (7755387.511606737 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19912906720580498 = (138898.32120977432 - 111239.62807081969) / 138898.32120977432 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.170623469387754 < 9.428188287619047.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/2/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 36.17729356077841 (T) = (0 -7215766.281329246) / Math.Sqrt((282822712613.15656 / (299)) + (37378666154.686714 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23659568719758894 = (9452090.013535036 - 7215766.281329246) / 9452090.013535036 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\b\w+n\b", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.61067333333333 < 31.49961086309524.
IsChangePoint: Marked as a change because one of 4/7/2023 3:57:10 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 111.30465992489826 (T) = (0 -23822769.47209653) / Math.Sqrt((1070726085135.0316 / (299)) + (50814700296.28778 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2879268548620382 = (33455508.93859499 - 23822769.47209653) / 33455508.93859499 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?s).*", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 59.43393568293488 < 67.38088981134938.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 5/2/2023 3:18:41 PM, 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 63.09464196628719 (T) = (0 -59.52116653510496) / Math.Sqrt((11.6898533197935 / (299)) + (0.10401959069315836 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18702494237645337 = (73.21401312002689 - 59.52116653510496) / 73.21401312002689 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\b\w+n\b", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.319603589743589 < 6.620480729166667.
IsChangePoint: Marked as a change because one of 5/10/2023 6:18:38 PM, 5/11/2023 9:33:40 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 10.825131423616648 (T) = (0 -6386149.189720634) / Math.Sqrt((65998395856.26665 / (299)) + (31386562941.726475 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08004898670310104 = (6941836.1384636145 - 6386149.189720634) / 6941836.1384636145 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 52.08163402777778 < 53.71439361197378.
IsChangePoint: Marked as a change because one of 3/25/2023 10:49:06 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.812579104725819 (T) = (0 -50821.44422360753) / Math.Sqrt((1543187.0433638387 / (299)) + (1743591.0871118512 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08599768667827673 = (55603.18992947525 - 50821.44422360753) / 55603.18992947525 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 44.05936617231637 < 50.25014922064164.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 47.381314887921626 (T) = (0 -44269.59814711606) / Math.Sqrt((1110672.789423877 / (299)) + (511528.30914882896 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1817380300143521 = (54101.98661425331 - 44269.59814711606) / 54101.98661425331 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 911.7041421568629 < 1.1186641827876982.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/15/2023 9:18:21 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 41.36581042594832 (T) = (0 -916518.5155944099) / Math.Sqrt((5020779175.115752 / (299)) + (22236453.044157036 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.15885608627928968 = (1089609.638308251 - 916518.5155944099) / 1089609.638308251 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 843.6206509539844 < 930.6439479166667.
IsChangePoint: Marked as a change because one of 3/28/2023 3:45:48 AM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 10.319540227257868 (T) = (0 -751929.4288514566) / Math.Sqrt((3255974384.6937356 / (299)) + (4544986378.447395 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20671329711828734 = (947865.9179839765 - 751929.4288514566) / 947865.9179839765 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 129.4457597730139 < 156.7487573474702.
IsChangePoint: Marked as a change because one of 3/28/2023 3:45:48 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 32.62931570564805 (T) = (0 -131408.18483453387) / Math.Sqrt((16686831.52923472 / (299)) + (13551063.47354426 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20648219522273836 = (165602.0621634568 - 131408.18483453387) / 165602.0621634568 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.20024393263992 < 39.81213076088344.
IsChangePoint: Marked as a change because one of 3/10/2023 12:53:36 AM, 4/25/2023 3:46:39 AM, 5/9/2023 1:22:51 PM, 5/16/2023 5:40:18 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 27.641525897605934 (T) = (0 -23229.756127385426) / Math.Sqrt((70512928.26611757 / (299)) + (2277.8016214891613 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.36627157166387897 = (36655.69522954188 - 23229.756127385426) / 36655.69522954188 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.StreamReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLine - Duration of single invocation 5.15 μs 4.43 μs 0.86 0.01 False
ReadLineAsync - Duration of single invocation 6.45 μs 5.88 μs 0.91 0.03 False
ReadLineAsync - Duration of single invocation 16.71 μs 13.88 μs 0.83 0.03 False
ReadLine - Duration of single invocation 21.73 μs 18.80 μs 0.87 0.01 False
ReadLine - Duration of single invocation 5.34 μs 4.92 μs 0.92 0.01 False
ReadLineAsync - Duration of single invocation 6.99 μs 6.32 μs 0.90 0.03 False
ReadLineAsync - Duration of single invocation 7.07 μs 6.35 μs 0.90 0.03 False
ReadLine - Duration of single invocation 5.26 μs 4.53 μs 0.86 0.02 False
ReadLine - Duration of single invocation 8.02 μs 7.06 μs 0.88 0.02 False
ReadLineAsync - Duration of single invocation 42.21 μs 36.66 μs 0.87 0.18 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 1024])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.4269367644074675 < 4.8840870061327815.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/11/2023 9:15:05 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 52.40438154075805 (T) = (0 -4398.626948802922) / Math.Sqrt((14690.466047774844 / (299)) + (1468.8141791667892 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13171309023077327 = (5065.868089583418 - 4398.626948802922) / 5065.868089583418 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [1025, 2048])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.881090887418366 < 6.291581135725653.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 62.00268980764267 (T) = (0 -5875.12885885805) / Math.Sqrt((16230.454933011657 / (299)) + (905.5549641853764 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10513716970318668 = (6565.396013721291 - 5875.12885885805) / 6565.396013721291 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 33, 128])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.883222526241681 < 16.594469791906214.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 32.12665613998917 (T) = (0 -14132.611304249158) / Math.Sqrt((262991.3142544077 / (299)) + (182498.84707291416 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21731299512969862 = (18056.52989804151 - 14132.611304249158) / 18056.52989804151 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 9, 32])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.801011440847322 < 20.66732796971446.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 41.661369588589686 (T) = (0 -19011.772619635743) / Math.Sqrt((1855297.388793156 / (299)) + (42685.38621369286 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1759079339948303 = (23069.961966502513 - 19011.772619635743) / 23069.961966502513 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [1025, 2048])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.917189260920525 < 5.01439769785947.
IsChangePoint: Marked as a change because one of 3/26/2023 5:20:11 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 54.834914770274004 (T) = (0 -4904.292840362432) / Math.Sqrt((4798.3348963497165 / (299)) + (629.6025670831906 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08238534438907932 = (5344.610409581241 - 4904.292840362432) / 5344.610409581241 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 129, 1024])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.316292654713708 < 6.671961799592826.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 37.91494041381954 (T) = (0 -6241.114328467921) / Math.Sqrt((14682.37693119997 / (299)) + (5867.684017718319 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11964463225910785 = (7089.312517606887 - 6241.114328467921) / 7089.312517606887 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 0, 1024])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.346006924198251 < 6.911005505141981.
IsChangePoint: Marked as a change because one of 4/11/2023 3:40:00 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 19.878478470976095 (T) = (0 -6405.895041346243) / Math.Sqrt((17217.226286606186 / (299)) + (26864.411815229396 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12512254349800478 = (7322.048355159136 - 6405.895041346243) / 7322.048355159136 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 129, 1024])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.526098018280102 < 5.002373402043895.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 7:05:50 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 42.562378841162634 (T) = (0 -4493.256523295763) / Math.Sqrt((11364.11404701528 / (299)) + (2974.7938607991005 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13400029305891606 = (5188.519681105909 - 4493.256523295763) / 5188.519681105909 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 33, 128])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.060512511220825 < 7.661174125949825.
IsChangePoint: Marked as a change because one of 4/25/2023 1:57:48 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 50.99679047198767 (T) = (0 -7046.963008821281) / Math.Sqrt((82405.72985006587 / (299)) + (5394.813486649919 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1597866848497655 = (8387.111798580872 - 7046.963008821281) / 8387.111798580872 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 9, 32])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 36.65529578003963 < 45.18687368349899.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 68.56519924884921 (T) = (0 -36422.62855326266) / Math.Sqrt((5733708.594173855 / (299)) + (65583.93018245806 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22658973157171408 = (47093.54147479867 - 36422.62855326266) / 47093.54147479867 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IterateForEachNonGeneric<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ArrayList - Duration of single invocation 3.02 μs 1.22 μs 0.40 0.06 True
Hashtable - Duration of single invocation 8.73 μs 6.86 μs 0.79 0.09 False
Queue - Duration of single invocation 3.59 μs 2.01 μs 0.56 0.12 True
Stack - Duration of single invocation 3.34 μs 1.37 μs 0.41 0.13 True
SortedList - Duration of single invocation 6.54 μs 4.38 μs 0.67 0.16 False

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEachNonGeneric<String>.ArrayList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2151562381494125 < 2.8717329029846983.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 105.27987485453498 (T) = (0 -1287.6252805426584) / Math.Sqrt((4300.630274602748 / (299)) + (3542.48048919746 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5806890444079145 = (3070.8123967914808 - 1287.6252805426584) / 3070.8123967914808 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateForEachNonGeneric<String>.Hashtable(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.8617464650267745 < 8.525561137563741.
IsChangePoint: Marked as a change because one of 5/10/2023 6:05:58 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 71.80853606335423 (T) = (0 -6827.194471175287) / Math.Sqrt((235326.24499272511 / (299)) + (14240.116263984062 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3133499249774677 = (9942.756462891602 - 6827.194471175287) / 9942.756462891602 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateForEachNonGeneric<String>.Queue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.0089270750842556 < 3.4148913663915303.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 271.1521112281201 (T) = (0 -2009.1139279113854) / Math.Sqrt((10952.085946180508 / (299)) + (2.230737648408003 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.45016317510321946 = (3654.018495921133 - 2009.1139279113854) / 3654.018495921133 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateForEachNonGeneric<String>.Stack(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3662706835311444 < 3.1720243779147315.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 265.7804986881863 (T) = (0 -1358.8309369910053) / Math.Sqrt((14182.119581614237 / (299)) + (151.1455127507993 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6005013008872958 = (3401.340079477105 - 1358.8309369910053) / 3401.340079477105 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateForEachNonGeneric<String>.SortedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.375484252873562 < 6.077062779504818.
IsChangePoint: Marked as a change because one of 5/10/2023 6:05:58 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 81.42193364761751 (T) = (0 -4384.056822873105) / Math.Sqrt((135777.5891102247 / (299)) + (10731.572452254286 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3991688805418793 = (7296.6540528510095 - 4384.056822873105) / 7296.6540528510095 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromReader - Duration of single invocation 34.69 μs 25.83 μs 0.74 0.06 True
DeserializeFromUtf8Bytes - Duration of single invocation 26.16 μs 21.97 μs 0.84 0.07 False
DeserializeFromString - Duration of single invocation 26.15 μs 20.68 μs 0.79 0.07 True
DeserializeFromStream - Duration of single invocation 28.81 μs 21.48 μs 0.75 0.07 True
DeserializeFromString - Duration of single invocation 27.26 μs 22.36 μs 0.82 0.07 True
DeserializeFromUtf8Bytes - Duration of single invocation 24.92 μs 19.79 μs 0.79 0.07 True
DeserializeFromStream - Duration of single invocation 29.35 μs 21.21 μs 0.72 0.10 True
DeserializeFromReader - Duration of single invocation 34.04 μs 25.74 μs 0.76 0.10 True

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromReader(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 25.828842114325063 < 32.91383337818385.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 61.47379954253845 (T) = (0 -25687.74623748319) / Math.Sqrt((590241.6139391374 / (299)) + (243566.45410051337 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25616948509626325 = (34534.40766786448 - 25687.74623748319) / 34534.40766786448 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.972705419580414 < 24.946932078872308.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 32.098522297605356 (T) = (0 -20281.916259689322) / Math.Sqrt((390690.33505924727 / (299)) + (374365.70059801824 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21543537258182363 = (25851.173441801082 - 20281.916259689322) / 25851.173441801082 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.682735539724476 < 25.174578539929975.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 53.33750992742677 (T) = (0 -20606.866406295565) / Math.Sqrt((428893.3999943425 / (299)) + (153063.83737230825 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22927149434258787 = (26736.868112485892 - 20606.866406295565) / 26736.868112485892 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.480301082640615 < 27.324232388105685.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 56.10961386441575 (T) = (0 -21127.78371745628) / Math.Sqrt((350992.7876177325 / (299)) + (199268.01088129115 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25437511426027604 = (28335.674038689984 - 21127.78371745628) / 28335.674038689984 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 22.357229244694132 < 25.75651796623008.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 23.269803269725216 (T) = (0 -21131.240786760838) / Math.Sqrt((279691.1142535383 / (299)) + (736625.6129536359 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20904001788080068 = (26715.941722038115 - 21131.240786760838) / 26715.941722038115 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.78638590391909 < 24.186586040366773.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 74.33117906439976 (T) = (0 -19687.668852535895) / Math.Sqrt((345442.56645240204 / (299)) + (73321.37874337718 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23735888284098092 = (25815.11593011946 - 19687.668852535895) / 25815.11593011946 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.209855712103206 < 26.941023716514866.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 46.12316252320041 (T) = (0 -21575.43591310823) / Math.Sqrt((667594.6931467605 / (299)) + (241164.05987747107 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2355866406435854 = (28224.828424340096 - 21575.43591310823) / 28224.828424340096 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromReader(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 25.74193939297934 < 32.31527313247676.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 67.80828526241775 (T) = (0 -25966.817309002006) / Math.Sqrt((908839.7703751331 / (299)) + (171152.8163009632 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24948765721088514 = (34598.787826062944 - 25966.817309002006) / 34598.787826062944 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToStream - Duration of single invocation 7.57 μs 5.80 μs 0.77 0.16 False
SerializeToUtf8Bytes - Duration of single invocation 9.29 μs 6.30 μs 0.68 0.46 False
SerializeToUtf8Bytes - Duration of single invocation 6.85 μs 5.96 μs 0.87 0.08 False
SerializeToStream - Duration of single invocation 10.15 μs 7.24 μs 0.71 0.47 False
SerializeToString - Duration of single invocation 8.81 μs 7.89 μs 0.90 0.35 False
SerializeObjectProperty - Duration of single invocation 7.54 μs 6.71 μs 0.89 0.18 False
SerializeToWriter - Duration of single invocation 8.50 μs 6.11 μs 0.72 0.35 False
SerializeObjectProperty - Duration of single invocation 9.46 μs 7.20 μs 0.76 0.32 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.797323580433285 < 7.548081836928935.
IsChangePoint: Marked as a change because one of 3/20/2023 4:51:16 PM, 4/25/2023 1:57:48 PM, 5/16/2023 12:30:38 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 45.51879796222245 (T) = (0 -5860.392591277818) / Math.Sqrt((160408.46740027188 / (299)) + (7981.595791143599 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2085164161118981 = (7404.3135076650515 - 5860.392591277818) / 7404.3135076650515 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.295297469920508 < 8.757686552705545.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 22.731945951504912 (T) = (0 -6560.879935025175) / Math.Sqrt((197665.94042724374 / (299)) + (135500.17720653434 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26727905561352916 = (8954.13183598675 - 6560.879935025175) / 8954.13183598675 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.9582693572712095 < 6.929537708646431.
IsChangePoint: Marked as a change because one of 5/1/2023 3:42:23 PM, 5/2/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 39.98104701926098 (T) = (0 -6028.548047800807) / Math.Sqrt((45997.30829889489 / (299)) + (11381.4928766395 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17544459317804387 = (7311.271016991262 - 6028.548047800807) / 7311.271016991262 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.2356875 < 9.079703595843442.
IsChangePoint: Marked as a change because one of 4/8/2023 9:16:19 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 25.921161978814638 (T) = (0 -7025.567195995215) / Math.Sqrt((244811.30195770273 / (299)) + (97574.05018884274 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25184909515724313 = (9390.575017043946 - 7025.567195995215) / 9390.575017043946 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.889715253201887 < 8.928969262033709.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 14.60428015001445 (T) = (0 -7125.54304724738) / Math.Sqrt((180120.46529563292 / (299)) + (271341.3876827036 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23097601292592085 = (9265.6967364023 - 7125.54304724738) / 9265.6967364023 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeObjectProperty(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.707646456197434 < 7.251124870611853.
IsChangePoint: Marked as a change because one of 3/20/2023 4:51:16 PM, 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 32.69238748600943 (T) = (0 -6778.825897530706) / Math.Sqrt((174422.20510876473 / (299)) + (19136.174051561553 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17941550396538403 = (8260.97242915097 - 6778.825897530706) / 8260.97242915097 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToWriter(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.111875 < 8.115169446715814.
IsChangePoint: Marked as a change because one of 3/25/2023 3:46:49 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 24.41610508220494 (T) = (0 -6216.318765148399) / Math.Sqrt((207962.5383384629 / (299)) + (110320.65639239959 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27344678680305107 = (8555.902929388494 - 6216.318765148399) / 8555.902929388494 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeObjectProperty(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.199247434094378 < 9.319751801305998.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 13.009359340237776 (T) = (0 -7402.962729180602) / Math.Sqrt((139282.81002673745 / (299)) + (363718.91691104387 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22862057514377387 = (9597.04458096015 - 7402.962729180602) / 9597.04458096015 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dequeue_And_Enqueue - Duration of single invocation 6.28 μs 4.64 μs 0.74 0.15 False
K_Max_Elements - Duration of single invocation 141.74 ns 55.51 ns 0.39 0.08 True
K_Max_Elements - Duration of single invocation 491.97 ns 239.76 ns 0.49 0.02 True
Dequeue_And_Enqueue - Duration of single invocation 168.59 μs 147.74 μs 0.88 0.01 False
HeapSort - Duration of single invocation 2.11 μs 1.97 μs 0.94 0.08 False
HeapSort - Duration of single invocation 151.80 ns 102.28 ns 0.67 0.04 True
Dequeue_And_Enqueue - Duration of single invocation 476.77 ns 215.98 ns 0.45 0.03 True
K_Max_Elements - Duration of single invocation 3.68 μs 1.93 μs 0.52 0.02 True
Enumerate - Duration of single invocation 2.99 μs 2.36 μs 0.79 0.00 True
Enumerate - Duration of single invocation 261.02 ns 246.68 ns 0.95 0.09 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Dequeue_And_Enqueue(Size: 100)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.637948936696341 < 5.990364805605097.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 24.815178363372844 (T) = (0 -4754.373812774798) / Math.Sqrt((55066.44123885935 / (299)) + (46959.59681560273 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24333996871114888 = (6283.3685092054775 - 4754.373812774798) / 6283.3685092054775 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.K_Max_Elements(Size: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 55.51366920698828 < 134.6932284102615.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 151.8609255898026 (T) = (0 -55.591088880353375) / Math.Sqrt((5.6142420483680135 / (299)) + (3.99558004150353 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6093810288475362 = (142.31538400795034 - 55.591088880353375) / 142.31538400795034 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.K_Max_Elements(Size: 100)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 239.7567753409643 < 467.4892561016881.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 104.99150326832907 (T) = (0 -243.18589007893868) / Math.Sqrt((18.21825544154835 / (299)) + (71.90279730196852 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5051783647812921 = (491.4617162433735 - 243.18589007893868) / 491.4617162433735 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Dequeue_And_Enqueue(Size: 1000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 147.73886006289308 < 160.44273661514336.
IsChangePoint: Marked as a change because one of 4/26/2023 8:04:03 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 20.979317914642216 (T) = (0 -146238.009900695) / Math.Sqrt((18357911.6497211 / (299)) + (5331087.350343663 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08967282618007642 = (160643.35340781894 - 146238.009900695) / 160643.35340781894 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.HeapSort(Size: 100)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.9692065540684092 < 2.0074071376059015.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/1/2023 3:42:23 PM, 5/16/2023 12:30:38 AM, 5/20/2023 3:57:43 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 2.9939876440248407 (T) = (0 -1799.5382285140415) / Math.Sqrt((10556.292488990997 / (299)) + (24607.825525474986 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.09019732823329399 = (1977.9434424165813 - 1799.5382285140415) / 1977.9434424165813 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.HeapSort(Size: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 102.2784741449043 < 144.2310215464447.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 59.730182196908814 (T) = (0 -101.71482613853975) / Math.Sqrt((7.759323404336479 / (299)) + (7.867867317205234 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.31812011962604475 = (149.16824658729848 - 101.71482613853975) / 149.16824658729848 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Dequeue_And_Enqueue(Size: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 215.98000000000002 < 453.7110706366181.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 374.7051723178792 (T) = (0 -212.65778755420308) / Math.Sqrt((14.733669515083957 / (299)) + (5.659730953195186 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5508934426300912 = (473.51298720638914 - 212.65778755420308) / 473.51298720638914 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.K_Max_Elements(Size: 1000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.9258396837102592 < 3.49228981616788.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 79.26919017610881 (T) = (0 -1991.913058519195) / Math.Sqrt((2615.023762064893 / (299)) + (5866.569047712551 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.460491088637138 = (3692.0855551530963 - 1991.913058519195) / 3692.0855551530963 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Enumerate(Size: 1000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.363898696772238 < 2.88372299436731.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 536.9795456699067 (T) = (0 -2364.0125552794507) / Math.Sqrt((440.9370416546342 / (299)) + (0.7608835873259946 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21952002296704817 = (3028.921464797094 - 2364.0125552794507) / 3028.921464797094 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Enumerate(Size: 100)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 246.68463978032787 < 248.98649118218867.
IsChangePoint: Marked as a change because one of 3/9/2023 8:37:21 PM, 5/19/2023 3:32:16 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 18.445748711521823 (T) = (0 -247.53831761625804) / Math.Sqrt((77.7305892537563 / (299)) + (9.671574343214983 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.07143806679677366 = (266.5824526774796 - 247.53831761625804) / 266.5824526774796 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Document.Tests.Perf_ParseThenWrite

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseThenWrite - Duration of single invocation 19.70 μs 14.17 μs 0.72 0.07 True
ParseThenWrite - Duration of single invocation 3.77 μs 2.80 μs 0.74 0.05 True
ParseThenWrite - Duration of single invocation 15.20 μs 11.89 μs 0.78 0.04 True
ParseThenWrite - Duration of single invocation 1.55 ms 1.16 ms 0.75 0.05 False
ParseThenWrite - Duration of single invocation 7.05 μs 4.72 μs 0.67 0.06 True
ParseThenWrite - Duration of single invocation 2.21 μs 1.69 μs 0.76 0.06 True
ParseThenWrite - Duration of single invocation 18.97 μs 14.92 μs 0.79 0.14 False
ParseThenWrite - Duration of single invocation 12.28 μs 9.27 μs 0.75 0.07 False
ParseThenWrite - Duration of single invocation 6.08 μs 4.18 μs 0.69 0.13 True
ParseThenWrite - Duration of single invocation 22.72 μs 17.01 μs 0.75 0.09 False
ParseThenWrite - Duration of single invocation 375.91 ns 317.86 ns 0.85 0.12 False
ParseThenWrite - Duration of single invocation 1.93 ms 1.47 ms 0.76 0.14 False
ParseThenWrite - Duration of single invocation 29.54 μs 20.81 μs 0.70 0.11 True
ParseThenWrite - Duration of single invocation 354.05 ns 288.56 ns 0.82 0.13 False
ParseThenWrite - Duration of single invocation 2.56 μs 1.93 μs 0.75 0.05 True
ParseThenWrite - Duration of single invocation 4.70 μs 3.48 μs 0.74 0.17 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_ParseThenWrite*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json4KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.165213443351798 < 18.69928213091378.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 74.02520678752454 (T) = (0 -14292.147781353355) / Math.Sqrt((268741.64154911856 / (299)) + (57182.17420710457 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2737714120869149 = (19679.957549486935 - 14292.147781353355) / 19679.957549486935 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfStrings)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.804688916726319 < 3.594438551649543.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 35.29706812359554 (T) = (0 -2803.5531893964444) / Math.Sqrt((12664.33175346046 / (299)) + (8003.3344592599715 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24411625939755224 = (3708.974064135817 - 2803.5531893964444) / 3708.974064135817 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json4KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.885097102573523 < 14.410663499244864.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 37.54729584993087 (T) = (0 -11587.329734349078) / Math.Sqrt((104144.82483893033 / (299)) + (114084.5510949662 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23636153754651748 = (15173.842471370968 - 11587.329734349078) / 15173.842471370968 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json400KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.162180527131783 < 1.477175405829052.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.66109783764141 (T) = (0 -1206905.8842194765) / Math.Sqrt((1505053317.7363458 / (299)) + (8367229261.05611 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21084796877540615 = (1529370.5603806388 - 1206905.8842194765) / 1529370.5603806388 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfNumbers)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.718189960438013 < 6.695403898621452.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 115.21695615222026 (T) = (0 -4616.414750377698) / Math.Sqrt((35845.18352488845 / (299)) + (3692.4793874567827 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.33404469142591753 = (6932.018847123818 - 4616.414750377698) / 6932.018847123818 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json400B)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.6850180156546737 < 2.0307022869464335.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 71.69855288604944 (T) = (0 -1694.5676765254757) / Math.Sqrt((3908.8823075416913 / (299)) + (316.27080011591784 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20556740171419016 = (2133.05405667131 - 1694.5676765254757) / 2133.05405667131 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: DeepTree)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.91574914647115 < 17.85754305509639.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 38.19503174670511 (T) = (0 -14254.258716782331) / Math.Sqrt((414358.798090752 / (299)) + (144978.54807964384 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23079184054536886 = (18531.081010488248 - 14254.258716782331) / 18531.081010488248 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: DeepTree)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.267867876767303 < 11.673458782780246.
IsChangePoint: Marked as a change because one of 4/17/2023 11:29:15 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 45.58634097803359 (T) = (0 -9127.031641276812) / Math.Sqrt((101361.67840960681 / (299)) + (35282.18444942081 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21628558911225565 = (11645.864251670786 - 9127.031641276812) / 11645.864251670786 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfNumbers)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.181878067450131 < 5.749656176545618.
IsChangePoint: Marked as a change because one of 3/23/2023 3:52:33 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 39.03421314509517 (T) = (0 -4012.172334914442) / Math.Sqrt((34112.045583488274 / (299)) + (26504.012255271275 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3110159385549925 = (5823.31661852918 - 4012.172334914442) / 5823.31661852918 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: BroadTree)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.01067267021422 < 21.55896325869728.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 19.0372238138761 (T) = (0 -17602.90407772418) / Math.Sqrt((380196.5444410946 / (299)) + (691270.9275225672 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20150116265114074 = (22044.996503900453 - 17602.90407772418) / 22044.996503900453 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: HelloWorld)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 317.8589174432069 < 359.80940021263035.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 34.0719973556368 (T) = (0 -311.00351947095794) / Math.Sqrt((268.6308645119386 / (299)) + (25.987750033658273 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15717411657943825 = (369.0009118001544 - 311.00351947095794) / 369.0009118001544 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json400KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.47461556122449 < 1.8537354571782942.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 15.82018544130605 (T) = (0 -1516871.36642879) / Math.Sqrt((4481526314.136725 / (299)) + (10322871983.985464 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22878495264084203 = (1966859.1421069314 - 1516871.36642879) / 1966859.1421069314 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: BroadTree)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.80980649685616 < 27.24145965955301.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 62.260868029053306 (T) = (0 -21239.254310132845) / Math.Sqrt((556209.3347544236 / (299)) + (182239.31368543315 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2697466791393199 = (29084.776067981664 - 21239.254310132845) / 29084.776067981664 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: HelloWorld)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 288.5555034209502 < 332.86628902023205.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 29.28086713880074 (T) = (0 -286.1065230535156) / Math.Sqrt((180.04950961541024 / (299)) + (49.71380114501819 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17716901329302331 = (347.70995219629805 - 286.1065230535156) / 347.70995219629805 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json400B)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.929177980276863 < 2.421685535582213.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 46.21256240048644 (T) = (0 -1958.1964294810443) / Math.Sqrt((2744.2455422340995 / (299)) + (2069.9244891853045 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23445086053669123 = (2557.897760624283 - 1958.1964294810443) / 2557.897760624283 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfStrings)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.4762308610422523 < 4.495719144712633.
IsChangePoint: Marked as a change because one of 4/8/2023 3:09:32 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 93.55480102751889 (T) = (0 -3483.0353567475995) / Math.Sqrt((33563.54089664142 / (299)) + (471.390967152227 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2466101346406817 = (4623.151328278642 - 3483.0353567475995) / 4623.151328278642 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 9.52 ns 6.90 ns 0.72 0.01 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.895919220284382 < 9.058093107232619.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 165.99129861466943 (T) = (0 -6.928372938098227) / Math.Sqrt((0.025132312534448656 / (299)) + (0.0018694951834293822 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2655975299308824 = (9.434027281316428 - 6.928372938098227) / 9.434027281316428 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Segment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadSingleSegmentSequenceByN - Duration of single invocation 735.45 μs 545.27 μs 0.74 0.24 False
ReadMultiSegmentSequence - Duration of single invocation 733.97 μs 618.23 μs 0.84 0.36 False
ReadSingleSegmentSequence - Duration of single invocation 81.21 μs 50.19 μs 0.62 0.28 False
ReadSingleSegmentSequenceByN - Duration of single invocation 68.90 μs 49.82 μs 0.72 0.23 False
ReadMultiSegmentSequence - Duration of single invocation 6.94 μs 6.38 μs 0.92 0.28 False
ReadMultiSegmentSequenceUsingSpan - Duration of single invocation 847.70 μs 561.28 μs 0.66 0.30 False
ReadSingleSegmentSequence - Duration of single invocation 731.39 μs 538.01 μs 0.74 0.32 False
ReadMultiSegmentSequenceUsingSpan - Duration of single invocation 84.35 μs 55.94 μs 0.66 0.31 False
ReadSingleSegmentSequenceByN - Duration of single invocation 7.75 μs 4.93 μs 0.64 0.28 False
ReadSingleSegmentSequenceByN - Duration of single invocation 77.75 μs 62.49 μs 0.80 0.30 False
ReadSingleSegmentSequence - Duration of single invocation 8.01 μs 5.18 μs 0.65 0.26 False
ReadMultiSegmentSequenceUsingSpan - Duration of single invocation 834.53 μs 558.97 μs 0.67 0.31 False
ReadSingleSegmentSequenceByN - Duration of single invocation 817.61 μs 503.56 μs 0.62 0.28 False
ReadMultiSegmentSequence - Duration of single invocation 72.23 μs 57.17 μs 0.79 0.40 False
ReadMultiSegmentSequenceUsingSpan - Duration of single invocation 7.78 μs 5.13 μs 0.66 0.21 False
ReadMultiSegmentSequenceUsingSpan - Duration of single invocation 77.59 μs 58.07 μs 0.75 0.32 False
ReadMultiSegmentSequence - Duration of single invocation 71.99 μs 58.44 μs 0.81 0.46 False
ReadMultiSegmentSequenceUsingSpan - Duration of single invocation 7.81 μs 5.19 μs 0.67 0.32 False
ReadMultiSegmentSequence - Duration of single invocation 734.13 μs 528.21 μs 0.72 0.43 False
ReadSingleSegmentSequenceByN - Duration of single invocation 7.11 μs 5.49 μs 0.77 0.30 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Segment*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 8192, TestCase: Json400KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 545.2699346405229 < 732.5688155009921.
IsChangePoint: Marked as a change because one of 3/11/2023 8:53:40 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 35.283545065572206 (T) = (0 -521660.8108262363) / Math.Sqrt((888454881.5306274 / (299)) + (453319926.6137615 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2938229434259613 = (738711.0724851808 - 521660.8108262363) / 738711.0724851808 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 8192, TestCase: Json400KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 618.2273800456968 < 690.4883942877933.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 16.109911457257468 (T) = (0 -552651.8224624901) / Math.Sqrt((1048667107.2151856 / (299)) + (2175308494.8566966 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27589148324976637 = (763216.851726543 - 552651.8224624901) / 763216.851726543 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase: Json40KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 50.18531335304822 < 70.2340789443817.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 50.06055998809309 (T) = (0 -52331.544287544486) / Math.Sqrt((12091631.712879403 / (299)) + (1983652.8065981027 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2959156964954896 = (74325.6795060895 - 52331.544287544486) / 74325.6795060895 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 8192, TestCase: Json40KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 49.82046841825902 < 68.46623611111112.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 28.30903946102373 (T) = (0 -52121.88292913017) / Math.Sqrt((7059345.453398857 / (299)) + (7540795.72077787 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.29676164269256494 = (74116.9510842595 - 52121.88292913017) / 74116.9510842595 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 8192, TestCase: Json4KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.383272339049953 < 6.675581035088564.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 5/19/2023 4:22:12 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 7.112746805535642 (T) = (0 -6734.8235762100785) / Math.Sqrt((114923.8724007306 / (299)) + (158294.20249353632 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.10248924213634183 = (7503.891755282083 - 6734.8235762100785) / 7503.891755282083 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 8192, TestCase: Json400KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 561.2844444444444 < 717.2480871841071.
IsChangePoint: Marked as a change because one of 3/17/2023 8:17:18 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 48.223539708300684 (T) = (0 -545177.4040303665) / Math.Sqrt((1153353678.7402534 / (299)) + (192338624.4718975 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27642395528674873 = (753448.6637771665 - 545177.4040303665) / 753448.6637771665 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase: Json400KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 538.00841954023 < 690.4628903544373.
IsChangePoint: Marked as a change because one of 3/8/2023 10:34:21 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 17.551450913573483 (T) = (0 -547191.296601644) / Math.Sqrt((1339563689.294805 / (299)) + (1599561535.985357 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26590250393210213 = (745393.2202910462 - 547191.296601644) / 745393.2202910462 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 4096, TestCase: Json40KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 55.935744904076756 < 73.11057508108965.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 26.53947357107455 (T) = (0 -55714.29596149809) / Math.Sqrt((12149737.344786081 / (299)) + (7117580.597211548 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26756774690228874 = (76067.50757611085 - 55714.29596149809) / 76067.50757611085 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 8192, TestCase: Json4KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.926240628604383 < 7.357892108626898.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 45.9973234590975 (T) = (0 -5000.97558033741) / Math.Sqrt((115853.96614257166 / (299)) + (32708.12733992162 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.33137510749186877 = (7479.493564138569 - 5000.97558033741) / 7479.493564138569 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 4096, TestCase: Json40KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 62.488003333333324 < 72.93219814243425.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 19.79699852453706 (T) = (0 -54070.14570958971) / Math.Sqrt((8498030.94530315 / (299)) + (13181294.258711703 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2720978844447151 = (74282.16590405421 - 54070.14570958971) / 74282.16590405421 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase: Json4KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.177191148931483 < 7.0974647050394015.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 46.63954279895061 (T) = (0 -4992.662917178974) / Math.Sqrt((106909.2265997918 / (299)) + (31952.52239026273 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3314047046620831 = (7467.391637351585 - 4992.662917178974) / 7467.391637351585 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 4096, TestCase: Json400KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 558.971215510813 < 747.1890303763439.
IsChangePoint: Marked as a change because one of 3/23/2023 6:11:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 20.069492908494798 (T) = (0 -538507.1317869268) / Math.Sqrt((1114022236.4836774 / (299)) + (1538600974.5732443 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2916750551403151 = (760254.3658738461 - 538507.1317869268) / 760254.3658738461 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 4096, TestCase: Json400KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 503.5589277389278 < 770.4715877604166.
IsChangePoint: Marked as a change because one of 3/8/2023 2:17:54 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 21.536338349105005 (T) = (0 -530916.8999255382) / Math.Sqrt((881512306.4334695 / (299)) + (1224775236.5012648 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2856357389798128 = (743201.9333768645 - 530916.8999255382) / 743201.9333768645 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 8192, TestCase: Json40KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 57.166145072992705 < 67.67460648837081.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 14.881650409946856 (T) = (0 -57043.107694139624) / Math.Sqrt((19913787.071438067 / (299)) + (20222513.659331493 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2494040754190561 = (75997.09221174718 - 57043.107694139624) / 75997.09221174718 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 4096, TestCase: Json4KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.133879423328965 < 7.1399270778479735.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 62.0630445599238 (T) = (0 -5301.671428125761) / Math.Sqrt((119477.5980958066 / (299)) + (19651.047235381393 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.33852351525211855 = (8014.90536756793 - 5301.671428125761) / 8014.90536756793 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 8192, TestCase: Json40KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 58.06656288723668 < 71.99094884762478.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 14.046241773926452 (T) = (0 -58749.72589161599) / Math.Sqrt((10660718.594311947 / (299)) + (16692155.995459588 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2154735605022706 = (74885.5907638101 - 58749.72589161599) / 74885.5907638101 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 4096, TestCase: Json40KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 58.4405006134133 < 69.65036431892636.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 17.520110337594655 (T) = (0 -56961.47425120126) / Math.Sqrt((13138771.28754723 / (299)) + (14282586.58768777 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2474290657079424 = (75689.17646917209 - 56961.47425120126) / 75689.17646917209 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 8192, TestCase: Json4KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.190935823481936 < 7.46536313370895.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 26.067958470184898 (T) = (0 -5176.471563285177) / Math.Sqrt((142834.95789449415 / (299)) + (118173.26260730236 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3300222598265694 = (7726.333656914026 - 5176.471563285177) / 7726.333656914026 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 4096, TestCase: Json400KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 528.2130750605328 < 709.1168278575167.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 36.61694620174653 (T) = (0 -545396.1656855118) / Math.Sqrt((1369270029.1384866 / (299)) + (409635668.16380733 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2874104418344619 = (765372.0987570429 - 545396.1656855118) / 765372.0987570429 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 4096, TestCase: Json4KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.494354019268403 < 7.083832270230535.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 38.66555232425203 (T) = (0 -5125.3376194412285) / Math.Sqrt((131142.4752976426 / (299)) + (41687.25666120924 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.31294143385086365 = (7459.826384478404 - 5125.3376194412285) / 7459.826384478404 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_FromStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractSerializer_ - Duration of single invocation 3.22 μs 2.79 μs 0.87 0.02 True
XmlSerializer_ - Duration of single invocation 4.24 μs 3.66 μs 0.86 0.00 True
DataContractSerializer_BinaryXml_ - Duration of single invocation 1.01 μs 777.52 ns 0.77 0.12 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;LoginViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_FromStream<LoginViewModel>.DataContractSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.790094691878006 < 3.1128138330719453.
IsChangePoint: Marked as a change because one of 4/25/2023 9:59:52 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 45.58777703330177 (T) = (0 -2792.966460288445) / Math.Sqrt((3460.6226612674704 / (299)) + (1780.7370934387047 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16593067383631965 = (3348.602295608632 - 2792.966460288445) / 3348.602295608632 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_FromStream<LoginViewModel>.XmlSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.661347820576565 < 4.030980317150783.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 51.76823731941665 (T) = (0 -3596.584976413933) / Math.Sqrt((983.8020448450728 / (299)) + (2186.677114559416 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15859990094702509 = (4274.524070608043 - 3596.584976413933) / 4274.524070608043 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_FromStream<LoginViewModel>.DataContractSerializer_BinaryXml_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 777.5237701741997 < 0.952290933263484.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 71.76500095083125 (T) = (0 -787.7749712240485) / Math.Sqrt((569.6872766379049 / (299)) + (105.34963412238348 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22372887555578863 = (1014.819367122632 - 787.7749712240485) / 1014.819367122632 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CtorFromCollection<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableHashSet - Duration of single invocation 139.09 μs 102.19 μs 0.73 0.01 True
ConcurrentBag - Duration of single invocation 12.48 μs 11.44 μs 0.92 0.07 False
HashSet - Duration of single invocation 10.60 μs 9.61 μs 0.91 0.03 False
ConcurrentQueue - Duration of single invocation 5.57 μs 4.71 μs 0.85 0.16 False
FrozenDictionaryOptimized - Duration of single invocation 132.52 μs 123.25 μs 0.93 0.09 False
ImmutableSortedSet - Duration of single invocation 294.63 μs 274.19 μs 0.93 0.05 False
ImmutableDictionary - Duration of single invocation 160.62 μs 129.90 μs 0.81 0.03 True
ImmutableStack - Duration of single invocation 5.17 μs 4.57 μs 0.88 0.18 False
LinkedList - Duration of single invocation 8.71 μs 6.90 μs 0.79 0.13 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<String>.ImmutableHashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 102.18667763157893 < 132.5171496000744.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 58.265555630880144 (T) = (0 -100852.4270534169) / Math.Sqrt((2037267.3570494975 / (299)) + (5680949.266682185 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2779151243852368 = (139668.3831212417 - 100852.4270534169) / 139668.3831212417 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<String>.ConcurrentBag(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.436341583508403 < 11.880916811665214.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 11.957133630453397 (T) = (0 -11644.142595366797) / Math.Sqrt((130005.52605262228 / (299)) + (39435.149176082115 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.055316450897751286 = (12325.97159803667 - 11644.142595366797) / 12325.97159803667 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<String>.HashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.605473775865601 < 10.075996428712603.
IsChangePoint: Marked as a change because one of 4/19/2023 8:38:27 PM, 4/25/2023 6:46:56 PM, 5/10/2023 6:18:38 PM, 5/11/2023 5:02:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 43.30350240496179 (T) = (0 -9610.145010787724) / Math.Sqrt((149198.38894820685 / (299)) + (7122.458237546441 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12724286057941095 = (11011.247661826934 - 9610.145010787724) / 11011.247661826934 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<String>.ConcurrentQueue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.711700487012988 < 5.396896292972006.
IsChangePoint: Marked as a change because one of 3/9/2023 2:48:42 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 51.92629653305845 (T) = (0 -4734.895032159765) / Math.Sqrt((37701.3715321856 / (299)) + (3380.5767762294367 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17729323303765834 = (5755.264478548404 - 4734.895032159765) / 5755.264478548404 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<String>.FrozenDictionaryOptimized(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 123.24657738095239 < 126.14395953619304.
IsChangePoint: Marked as a change because one of 4/21/2023 8:59:32 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.651590790448534 (T) = (0 -122080.5160107281) / Math.Sqrt((16615352.358611314 / (265)) + (18912671.045958523 / (13))) is greater than 1.968596344330496 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (265) + (13) - 2, .975) and 0.11321046937050863 = (137665.71637812268 - 122080.5160107281) / 137665.71637812268 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<String>.ImmutableSortedSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 274.18815789473683 < 306.81228186054926.
IsChangePoint: Marked as a change because one of 4/25/2023 9:59:52 PM, 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 7.9615427373620316 (T) = (0 -278599.5779521215) / Math.Sqrt((166340376.04250833 / (299)) + (291781351.09252006 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1205340033921475 = (316782.66019004147 - 278599.5779521215) / 316782.66019004147 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<String>.ImmutableDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 129.89930222088836 < 151.7846973556175.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 39.97863674817188 (T) = (0 -129560.03693501544) / Math.Sqrt((6058528.520651408 / (299)) + (7909913.813614972 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19657564187661689 = (161259.7821127035 - 129560.03693501544) / 161259.7821127035 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<String>.ImmutableStack(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.568052179783951 < 4.747116762770103.
IsChangePoint: Marked as a change because one of 4/12/2023 9:47:42 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 44.7610084359633 (T) = (0 -4537.790187641269) / Math.Sqrt((50143.22242292945 / (299)) + (1715.3157661676435 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1458473124391019 = (5312.621799036066 - 4537.790187641269) / 5312.621799036066 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<String>.LinkedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.899276125503018 < 8.249259866601486.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 5/20/2023 7:45:57 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 18.149412259668363 (T) = (0 -6804.091332856974) / Math.Sqrt((147484.8019731657 / (299)) + (60368.189240440144 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.1930062182028049 = (8431.404908355173 - 6804.091332856974) / 8431.404908355173 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Basic

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteBasicUtf8 - Duration of single invocation 789.94 ns 676.21 ns 0.86 0.07 False
WriteBasicUtf8 - Duration of single invocation 624.63 ns 569.12 ns 0.91 0.09 False
WriteBasicUtf8 - Duration of single invocation 660.63 ns 604.56 ns 0.92 0.04 False
WriteBasicUtf16 - Duration of single invocation 760.79 ns 669.46 ns 0.88 0.05 False
WriteBasicUtf16 - Duration of single invocation 887.19 ns 824.89 ns 0.93 0.04 False
WriteBasicUtf8 - Duration of single invocation 828.12 ns 693.48 ns 0.84 0.08 False
WriteBasicUtf16 - Duration of single invocation 732.31 ns 621.93 ns 0.85 0.08 False
WriteBasicUtf16 - Duration of single invocation 916.78 ns 829.66 ns 0.90 0.06 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Basic*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: True, DataSize: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 676.2144748713495 < 744.0553404142267.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 12:29:35 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 40.32526523911083 (T) = (0 -677.4435066646886) / Math.Sqrt((1369.9862341129458 / (299)) + (58.79152787724613 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1522614061018722 = (799.118397511694 - 677.4435066646886) / 799.118397511694 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: True, DataSize: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 569.1195420292632 < 588.1215119471158.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 12:29:35 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 19.169393941365833 (T) = (0 -564.2172383254067) / Math.Sqrt((969.1607182848949 / (299)) + (101.85724426640917 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10158730500402054 = (628.0156563548244 - 564.2172383254067) / 628.0156563548244 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: False, DataSize: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 604.563522098018 < 631.2409252994139.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 12:29:35 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 23.761826644896235 (T) = (0 -602.4350864065813) / Math.Sqrt((876.9670066963133 / (299)) + (88.3840056853856 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10956406908369593 = (676.5619686827381 - 602.4350864065813) / 676.5619686827381 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: False, DataSize: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 669.4647666392294 < 714.4280820328861.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 12:29:35 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 37.388880729951694 (T) = (0 -668.2082074651224) / Math.Sqrt((1100.0864059005141 / (299)) + (73.34918835359599 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1459075040519806 = (782.3604710675154 - 668.2082074651224) / 782.3604710675154 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: True, DataSize: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 824.893142472936 < 841.2223319401327.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 12:29:35 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 31.21495466951098 (T) = (0 -806.8007653052404) / Math.Sqrt((1109.8736017404763 / (299)) + (83.86368522135449 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10979836087626998 = (906.3123789565415 - 806.8007653052404) / 906.3123789565415 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: False, DataSize: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 693.482312837664 < 777.1320014061899.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 12:29:35 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 44.39676978942563 (T) = (0 -698.8180263493541) / Math.Sqrt((1360.3619590171575 / (299)) + (55.834136644415764 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15891571659351664 = (830.8537445487207 - 698.8180263493541) / 830.8537445487207 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: True, DataSize: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 621.9277978533811 < 695.3284596992124.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 12:29:35 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 23.147313733216276 (T) = (0 -643.7291441201655) / Math.Sqrt((964.9414237900528 / (299)) + (210.4264968661156 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13676709179709048 = (745.7189571934763 - 643.7291441201655) / 745.7189571934763 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: False, DataSize: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 829.6606479808104 < 871.7608798378606.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 12:29:35 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 17.57777448139108 (T) = (0 -848.4445070695818) / Math.Sqrt((1024.9207852454854 / (299)) + (289.6456140189101 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09505975591229004 = (937.5696490599966 - 848.4445070695818) / 937.5696490599966 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tests.Perf_Timer

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ShortScheduleAndDisposeWithFiringTimers - Duration of single invocation 133.44 ns 110.42 ns 0.83 0.29 False
ShortScheduleAndDispose - Duration of single invocation 121.11 ns 97.84 ns 0.81 0.10 False
LongScheduleAndDispose - Duration of single invocation 120.81 ns 93.45 ns 0.77 0.06 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_Timer*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_Timer.ShortScheduleAndDisposeWithFiringTimers

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 110.4187847915135 < 130.84943310524872.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 20.833994218187495 (T) = (0 -118.05725768361965) / Math.Sqrt((74.5347699222822 / (299)) + (23.58709473788763 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20224200304672257 = (147.98630428587725 - 118.05725768361965) / 147.98630428587725 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Threading.Tests.Perf_Timer.ShortScheduleAndDispose

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 97.83680264580389 < 120.4655057359253.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 23.59139526758489 (T) = (0 -99.46450530106921) / Math.Sqrt((45.35413020479716 / (299)) + (25.79517735560222 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25741120482195584 = (133.94291153722762 - 99.46450530106921) / 133.94291153722762 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Threading.Tests.Perf_Timer.LongScheduleAndDispose

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 93.45479694356867 < 116.37034514640392.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 24.182821253387182 (T) = (0 -97.78880953271008) / Math.Sqrt((16.749535872676695 / (299)) + (22.859220353238403 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24987394740937374 = (130.36316922334296 - 97.78880953271008) / 130.36316922334296 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringWithCultureInfo - Duration of single invocation 108.16 ns 100.34 ns 0.93 0.03 False
ToStringWithFormat - Duration of single invocation 199.38 ns 170.56 ns 0.86 0.05 False
Parse - Duration of single invocation 57.75 ns 43.35 ns 0.75 0.05 False
ToStringWithFormat - Duration of single invocation 299.63 ns 258.08 ns 0.86 0.07 False
ToStringWithCultureInfo - Duration of single invocation 195.61 ns 171.74 ns 0.88 0.03 False
ToStringWithFormat - Duration of single invocation 176.95 ns 163.54 ns 0.92 0.04 False
Parse - Duration of single invocation 128.11 ns 93.73 ns 0.73 0.10 False
ToStringWithFormat - Duration of single invocation 134.46 ns 119.87 ns 0.89 0.07 False
TryParse - Duration of single invocation 120.57 ns 86.98 ns 0.72 0.10 False
ToStringWithFormat - Duration of single invocation 206.46 ns 171.99 ns 0.83 0.02 False
ToString - Duration of single invocation 186.49 ns 170.38 ns 0.91 0.06 False
ToStringWithFormat - Duration of single invocation 171.23 ns 151.44 ns 0.88 0.07 False
TryParse - Duration of single invocation 57.19 ns 44.23 ns 0.77 0.03 False
ToStringWithFormat - Duration of single invocation 173.33 ns 155.22 ns 0.90 0.04 False
ToStringWithFormat - Duration of single invocation 205.40 ns 170.43 ns 0.83 0.05 False
ToStringWithFormat - Duration of single invocation 134.14 ns 116.78 ns 0.87 0.04 False
ToStringWithCultureInfo - Duration of single invocation 193.59 ns 165.86 ns 0.86 0.07 False
ToString - Duration of single invocation 113.37 ns 103.00 ns 0.91 0.03 False
TryParse - Duration of single invocation 126.55 ns 83.50 ns 0.66 0.11 False
Parse - Duration of single invocation 120.63 ns 84.82 ns 0.70 0.09 False
ToStringWithFormat - Duration of single invocation 135.95 ns 115.42 ns 0.85 0.04 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.ToStringWithCultureInfo(value: 12345, culture: zh)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 100.34145216615798 < 102.9032435507327.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 28.409726094403585 (T) = (0 -100.12859590761825) / Math.Sqrt((6.005486681148612 / (299)) + (1.4189139682014777 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09255793311894642 = (110.3415849474201 - 100.12859590761825) / 110.3415849474201 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 170.56124571641286 < 173.89170501245803.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/11/2023 5:02:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 35.84002005863278 (T) = (0 -173.11589189912652) / Math.Sqrt((50.99047476458842 / (299)) + (3.157550236077127 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11747770570128552 = (196.16036106678862 - 173.11589189912652) / 196.16036106678862 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 43.34769560165859 < 54.74175811491853.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/15/2023 2:53:22 PM, 4/10/2023 1:33:42 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 52.46949295365372 (T) = (0 -44.32598441470896) / Math.Sqrt((6.654151585755161 / (299)) + (0.9605360821595981 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2684888169521439 = (60.59508787004985 - 44.32598441470896) / 60.59508787004985 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "F50")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 258.07687468139613 < 258.4849137450504.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/12/2023 8:39:35 PM, 5/16/2023 5:40:18 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 7.7842906186829595 (T) = (0 -256.50204240197183) / Math.Sqrt((210.59000215001765 / (299)) + (94.90888052961667 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.060487985202256525 = (273.01624498883194 - 256.50204240197183) / 273.01624498883194 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithCultureInfo(value: -1.7976931348623157E+308, culture: zh)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 171.74410076960476 < 185.20130202279339.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 29.465592266282247 (T) = (0 -171.94973601989088) / Math.Sqrt((25.663991784312387 / (299)) + (4.1288217447362365 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09815899725352961 = (190.66524531068606 - 171.94973601989088) / 190.66524531068606 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G17")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 163.53677428333623 < 178.04290135279288.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/14/2023 5:56:01 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 20.3215130469493 (T) = (0 -161.62519519636245) / Math.Sqrt((29.1811901339071 / (299)) + (19.190269420661856 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13624166236200644 = (187.11853553661507 - 161.62519519636245) / 187.11853553661507 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.Parse(value: "-1.7976931348623157e+308")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 93.73412018450588 < 118.08824753963098.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 37.71198523559306 (T) = (0 -88.984233213014) / Math.Sqrt((78.87851256394407 / (299)) + (13.709858487388138 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.327334154112419 = (132.28593923272484 - 88.984233213014) / 132.28593923272484 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "E")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 119.86898979157327 < 136.34224962810552.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/11/2023 5:52:15 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 27.873627734519836 (T) = (0 -115.85093041790341) / Math.Sqrt((32.62853348227194 / (299)) + (7.159175002510892 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16348696721381095 = (138.4926783890463 - 115.85093041790341) / 138.4926783890463 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.TryParse(value: "1.7976931348623157e+308")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 86.98419476919963 < 113.8759637948632.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 46.73856458129892 (T) = (0 -85.86205934724127) / Math.Sqrt((98.7874381733346 / (299)) + (6.205932846640447 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3285148159829577 = (127.86888138556768 - 85.86205934724127) / 127.86888138556768 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "R")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 171.99281496062994 < 194.9173618192796.
IsChangePoint: Marked as a change because one of 3/17/2023 9:25:32 PM, 4/11/2023 5:52:15 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 60.49652873367178 (T) = (0 -172.94537829019504) / Math.Sqrt((37.171241810466555 / (299)) + (1.3587688715210324 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.143347863343922 = (201.88518873633276 - 172.94537829019504) / 201.88518873633276 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToString(value: -1.7976931348623157E+308)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 170.37893227262842 < 179.03317031924777.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/24/2023 11:08:13 AM, 5/5/2023 3:18:41 PM, 5/11/2023 5:02:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 24.919151817790542 (T) = (0 -170.50787111095266) / Math.Sqrt((81.54507061045477 / (299)) + (9.298327349224783 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12684023600710867 = (195.27683036061293 - 170.50787111095266) / 195.27683036061293 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G17")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 151.44180335365033 < 169.74769739049287.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/16/2023 8:14:39 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 30.96065351838452 (T) = (0 -153.88915463492873) / Math.Sqrt((85.83782082291535 / (299)) + (4.304484272856202 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13658016663836528 = (178.23212843719077 - 153.88915463492873) / 178.23212843719077 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 44.229210778003676 < 57.448136971695114.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 5/16/2023 12:30:38 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 62.47088625312921 (T) = (0 -44.822124421332745) / Math.Sqrt((7.419504142309638 / (299)) + (0.5034997949132728 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2599932670192438 = (60.569887304658266 - 44.822124421332745) / 60.569887304658266 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G17")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 155.21947077026195 < 171.6447868244766.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/16/2023 8:14:39 PM, 4/12/2023 8:39:35 PM, 4/25/2023 1:57:48 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 30.216560246754458 (T) = (0 -153.6463746288258) / Math.Sqrt((85.97524167152126 / (299)) + (6.4156363033346215 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14806994838787343 = (180.35092709557233 - 153.6463746288258) / 180.35092709557233 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "R")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 170.4312860262078 < 192.01139571650373.
IsChangePoint: Marked as a change because one of 3/6/2023 6:03:21 AM, 3/17/2023 10:06:19 AM, 4/11/2023 5:52:15 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 15.677530486362668 (T) = (0 -170.8454087776122) / Math.Sqrt((32.155330905660236 / (299)) + (42.05531740920775 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14366702297414896 = (199.5081508725486 - 170.8454087776122) / 199.5081508725486 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "E")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 116.78264803102643 < 131.77609366614104.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/12/2023 4:35:31 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 30.793580610509522 (T) = (0 -115.31439618975472) / Math.Sqrt((44.46752010181604 / (299)) + (6.010008121657908 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17269286194056235 = (139.38523056895224 - 115.31439618975472) / 139.38523056895224 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithCultureInfo(value: 1.7976931348623157E+308, culture: zh)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 165.85994600241688 < 183.49225272742763.
IsChangePoint: Marked as a change because one of 3/6/2023 8:17:12 PM, 4/24/2023 11:08:13 AM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 8.351492850294347 (T) = (0 -177.03933597639266) / Math.Sqrt((47.74421675433204 / (299)) + (41.56838760923922 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.05703925590623425 = (187.74836289344861 - 177.03933597639266) / 187.74836289344861 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToString(value: 12345)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 102.99955117511854 < 107.67839896963152.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/22/2023 10:50:22 PM, 4/10/2023 1:33:42 PM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.535737966802303 (T) = (0 -106.03787176655283) / Math.Sqrt((13.489458060270803 / (299)) + (12.575027449126523 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.07673229459735258 = (114.85062365558267 - 106.03787176655283) / 114.85062365558267 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.TryParse(value: "-1.7976931348623157e+308")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 83.49999629525925 < 120.60170821879427.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 36.3191933202322 (T) = (0 -86.72940883795667) / Math.Sqrt((88.28376412296222 / (299)) + (16.436955590968008 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.34339128909310296 = (132.0868995450387 - 86.72940883795667) / 132.0868995450387 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.Parse(value: "1.7976931348623157e+308")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 84.82333323494926 < 114.04657569683906.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 46.16086918733067 (T) = (0 -86.46986111110606) / Math.Sqrt((80.75255133232552 / (299)) + (7.02588372557554 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.32460291417663806 = (128.02818212591563 - 86.46986111110606) / 128.02818212591563 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "E")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 115.41682535007793 < 137.19005643693436.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/10/2023 6:18:38 PM, 5/11/2023 5:02:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 37.514434767356285 (T) = (0 -116.97037967991993) / Math.Sqrt((31.28531761887279 / (299)) + (3.814183882970808 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16828841319376897 = (140.63815093533287 - 116.97037967991993) / 140.63815093533287 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToString<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
JsonNet_ - Duration of single invocation 746.91 ns 440.65 ns 0.59 0.08 True
SystemTextJson_SourceGen_ - Duration of single invocation 263.53 ns 222.84 ns 0.85 0.08 False
SystemTextJson_Reflection_ - Duration of single invocation 360.13 ns 277.74 ns 0.77 0.05 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;LoginViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToString<LoginViewModel>.JsonNet_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 440.6466829337232 < 696.1573276984241.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 124.67829122756059 (T) = (0 -439.00706693591303) / Math.Sqrt((307.04061046402694 / (299)) + (51.848112275493726 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.38875671630899206 = (718.219862122587 - 439.00706693591303) / 718.219862122587 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToString<LoginViewModel>.SystemTextJson_SourceGen_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 222.84086043024868 < 251.46825060941046.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 35.078721659789444 (T) = (0 -224.3765198102092) / Math.Sqrt((40.99643916403053 / (299)) + (19.7880755926634 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16762670737054855 = (269.56237279238985 - 224.3765198102092) / 269.56237279238985 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToString<LoginViewModel>.SystemTextJson_Reflection_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 277.7375620881256 < 352.50690782889995.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 65.27014443193106 (T) = (0 -284.330332033403) / Math.Sqrt((88.84763607117726 / (299)) + (20.30342650855015 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2383774562391256 = (373.3218434283555 - 284.330332033403) / 373.3218434283555 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse_Flags - Duration of single invocation 87.44 ns 80.35 ns 0.92 0.15 False
ToString_Format_NonFlags - Duration of single invocation 10.20 ns 8.90 ns 0.87 0.16 False
ToString_Format_NonFlags - Duration of single invocation 20.97 ns 18.65 ns 0.89 0.11 False
ToString_Flags - Duration of single invocation 23.38 ns 18.89 ns 0.81 0.10 False
ToString_Format_NonFlags - Duration of single invocation 10.23 ns 8.86 ns 0.87 0.17 False
GetName_NonGeneric_Flags - Duration of single invocation 44.67 ns 32.36 ns 0.72 0.10 False
ToString_Format_Flags_Large - Duration of single invocation 20.89 ns 18.66 ns 0.89 0.11 False
ToString_Format_NonFlags - Duration of single invocation 14.85 ns 12.51 ns 0.84 0.07 False
InterpolateIntoSpan_Flags - Duration of single invocation 122.96 ns 103.44 ns 0.84 0.11 False
InterpolateIntoStringBuilder_NonFlags - Duration of single invocation 107.62 ns 100.80 ns 0.94 0.25 False
ToString_Format_NonFlags - Duration of single invocation 20.90 ns 18.67 ns 0.89 0.10 False
GetName_Generic_Flags - Duration of single invocation 6.69 ns 5.61 ns 0.84 0.03 False
ToString_Format_Flags_Large - Duration of single invocation 20.99 ns 16.13 ns 0.77 0.11 False
GetValues_Generic - Duration of single invocation 50.00 ns 40.73 ns 0.81 0.25 False
ToString_Format_NonFlags - Duration of single invocation 32.84 ns 25.05 ns 0.76 0.08 False
ToString_NonFlags_Large - Duration of single invocation 12.08 ns 10.49 ns 0.87 0.13 False
ToString_Flags - Duration of single invocation 26.02 ns 20.16 ns 0.77 0.12 False
InterpolateIntoSpan_Flags - Duration of single invocation 86.00 ns 80.61 ns 0.94 0.05 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.Parse_Flags(text: "Red")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 80.34595504459637 < 84.97705925961986.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/18/2023 4:36:42 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 13.616094141362801 (T) = (0 -80.2410329626881) / Math.Sqrt((12.86058758009183 / (299)) + (2.893125719378805 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08041368617010297 = (87.25775031220277 - 80.2410329626881) / 87.25775031220277 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_Format_NonFlags(value: Tuesday, format: "d")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.902462975590824 < 9.870793512071026.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 3/31/2023 8:05:26 PM, 4/19/2023 1:39:34 AM, 4/25/2023 6:46:56 PM, 5/1/2023 3:42:23 PM, 5/16/2023 7:34:13 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.823554256152558 (T) = (0 -9.470339406190691) / Math.Sqrt((1.5223969893610918 / (299)) + (0.16545762969652905 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15308303277368632 = (11.182134462610218 - 9.470339406190691) / 11.182134462610218 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_Format_NonFlags(value: Friday, format: "X")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.647666491915817 < 19.913144814352584.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 4/25/2023 1:57:48 PM, 5/16/2023 12:30:38 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 24.773470975817883 (T) = (0 -18.885363865885523) / Math.Sqrt((1.46160727046133 / (299)) + (0.2226842401721576 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16293311801244145 = (22.561355934956485 - 18.885363865885523) / 22.561355934956485 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_Flags(value: 32)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.88933958162526 < 22.077587131789375.
IsChangePoint: Marked as a change because one of 4/19/2023 8:38:27 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 28.89602788031673 (T) = (0 -18.495996695829145) / Math.Sqrt((2.6344223056260865 / (299)) + (0.10913007620714597 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17007212045023126 = (22.28626986945314 - 18.495996695829145) / 22.28626986945314 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_Format_NonFlags(value: Saturday, format: "D")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.861998292229181 < 9.914170996170483.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 3/31/2023 8:05:26 PM, 4/19/2023 1:39:34 AM, 4/25/2023 6:46:56 PM, 5/1/2023 3:42:23 PM, 5/16/2023 12:30:38 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 14.283719966884759 (T) = (0 -9.378920242577303) / Math.Sqrt((1.51592687036805 / (299)) + (0.138306891380593 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16028547225567943 = (11.169177062794645 - 9.378920242577303) / 11.169177062794645 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.GetName_NonGeneric_Flags

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 32.36084012545762 < 43.45970970238343.
IsChangePoint: Marked as a change because one of 3/6/2023 6:03:21 AM, 3/17/2023 10:06:19 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 36.03002654087162 (T) = (0 -30.993101427474564) / Math.Sqrt((1.9844543315184073 / (299)) + (1.9908934347902056 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3172621652249501 = (45.395318449996616 - 30.993101427474564) / 45.395318449996616 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_Format_Flags_Large(value: All, format: "x")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.66023544256998 < 19.844189123848523.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 4/25/2023 1:57:48 PM, 5/16/2023 12:30:38 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 25.540747654169884 (T) = (0 -18.976045751702586) / Math.Sqrt((1.200421634521747 / (299)) + (0.20344421635722856 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.158774280119033 = (22.557614803298794 - 18.976045751702586) / 22.557614803298794 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_Format_NonFlags(value: Sunday, format: "")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.51254365114703 < 14.156096648663919.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 7.378253771442537 (T) = (0 -12.969301094175288) / Math.Sqrt((1.5434956935806927 / (299)) + (0.12107623999546567 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06406257130488714 = (13.857017249814586 - 12.969301094175288) / 13.857017249814586 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.InterpolateIntoSpan_Flags(value: Red, Green)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 103.44339900400416 < 111.39623492372093.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 4/6/2023 9:09:21 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 28.85300434592492 (T) = (0 -103.05717654459309) / Math.Sqrt((18.73129006162853 / (299)) + (0.10615587507728745 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06933618677600724 = (110.7351280669051 - 103.05717654459309) / 110.7351280669051 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.InterpolateIntoStringBuilder_NonFlags(value: Saturday)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 100.80469921268936 < 101.40532693395699.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/9/2023 1:22:51 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 25.101454401189155 (T) = (0 -99.8581758243886) / Math.Sqrt((34.11309216618115 / (299)) + (4.960619273985553 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15036504377506468 = (117.53068196260959 - 99.8581758243886) / 117.53068196260959 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_Format_NonFlags(value: Wednesday, format: "x")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.673023514929323 < 19.864430287743087.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 4/25/2023 1:57:48 PM, 5/16/2023 12:30:38 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 24.65111280059626 (T) = (0 -19.013988252801767) / Math.Sqrt((2.3093917378611666 / (299)) + (0.17350208360087746 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15838356565657624 = (22.59222548052461 - 19.013988252801767) / 22.59222548052461 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.GetName_Generic_Flags

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.608369420888399 < 6.159448691004623.
IsChangePoint: Marked as a change because one of 3/17/2023 9:25:32 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 36.89984438417749 (T) = (0 -5.617389499351224) / Math.Sqrt((0.24911869818322935 / (299)) + (0.00016631681908085228 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16041074395361216 = (6.6906400467812315 - 5.617389499351224) / 6.6906400467812315 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_Format_Flags_Large(value: All, format: "d")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.13123476572757 < 20.04743492013304.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/25/2023 12:18:49 AM, 4/9/2023 11:04:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 15.584826997964438 (T) = (0 -16.758020690787195) / Math.Sqrt((2.4681049245508655 / (299)) + (0.2973281619263745 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14094826688571 = (19.50758032934167 - 16.758020690787195) / 19.50758032934167 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.GetValues_Generic

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 40.731171309845934 < 44.626551413502504.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 4/22/2023 6:52:03 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 18.01796731474353 (T) = (0 -38.92657217215292) / Math.Sqrt((8.278094394802448 / (299)) + (1.4403788785115184 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14693984441077973 = (45.63168484321696 - 38.92657217215292) / 45.63168484321696 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_Format_NonFlags(value: 8, format: "F")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 25.05222572838051 < 31.066202007730343.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 14.63857084766509 (T) = (0 -25.01748079907363) / Math.Sqrt((5.422623351192672 / (299)) + (0.28231488451210623 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10459303769769192 = (27.939788110143382 - 25.01748079907363) / 27.939788110143382 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_NonFlags_Large(value: 42)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.488148788867152 < 11.56222151092854.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/9/2023 11:04:55 PM, 4/19/2023 1:39:34 AM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 11.891624787407226 (T) = (0 -10.52947193868416) / Math.Sqrt((0.5318875451146219 / (299)) + (0.13092805892429463 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10948173996882186 = (11.82398206895332 - 10.52947193868416) / 11.82398206895332 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_Flags(value: 36)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.156975138906805 < 24.55852170955712.
IsChangePoint: Marked as a change because one of 3/20/2023 4:51:16 PM, 3/25/2023 12:18:49 AM, 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 18.672646926104143 (T) = (0 -20.4272355959858) / Math.Sqrt((2.831864919047354 / (299)) + (0.14996655366051767 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11698887680016164 = (23.133610731835386 - 20.4272355959858) / 23.133610731835386 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.InterpolateIntoSpan_Flags(value: Red)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 80.60617609078875 < 81.7127782684773.
IsChangePoint: Marked as a change because one of 3/22/2023 1:04:34 PM, 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 18.875355457653672 (T) = (0 -81.29001515841337) / Math.Sqrt((10.272361008374808 / (299)) + (1.175077468328458 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.07395439809864796 = (87.78187055962377 - 81.29001515841337) / 87.78187055962377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 69.51 ns 51.55 ns 0.74 0.08 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 51.54705660186913 < 65.89106407410196.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/10/2023 1:33:42 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 63.430021919620316 (T) = (0 -48.96999661833499) / Math.Sqrt((9.561290389054923 / (299)) + (0.8010220611763108 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.28380571241397917 = (68.37529629479667 - 48.96999661833499) / 68.37529629479667 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeObjectProperty - Duration of single invocation 545.37 ns 429.50 ns 0.79 0.06 True
SerializeToWriter - Duration of single invocation 168.56 ns 126.28 ns 0.75 0.11 False
SerializeToStream - Duration of single invocation 465.34 ns 322.95 ns 0.69 0.07 True
SerializeToUtf8Bytes - Duration of single invocation 314.01 ns 251.91 ns 0.80 0.09 False
SerializeToString - Duration of single invocation 279.54 ns 231.15 ns 0.83 0.11 False
SerializeToString - Duration of single invocation 361.84 ns 280.09 ns 0.77 0.09 False
SerializeToUtf8Bytes - Duration of single invocation 245.44 ns 204.52 ns 0.83 0.07 False
SerializeToWriter - Duration of single invocation 257.09 ns 195.92 ns 0.76 0.16 False
SerializeObjectProperty - Duration of single invocation 493.97 ns 380.36 ns 0.77 0.07 True
SerializeToStream - Duration of single invocation 312.68 ns 267.63 ns 0.86 0.12 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LoginViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeObjectProperty(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 429.50445249952855 < 531.6457548192369.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 56.222320578530564 (T) = (0 -428.3741268971728) / Math.Sqrt((213.41265938410277 / (299)) + (62.47763929376881 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.235678684038311 = (560.4634045279521 - 428.3741268971728) / 560.4634045279521 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToWriter(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 126.28469759450172 < 160.23869380594266.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 19.45152333289759 (T) = (0 -129.35123704451183) / Math.Sqrt((59.450698650015056 / (299)) + (73.22145249667037 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2663950541618565 = (176.322743975953 - 129.35123704451183) / 176.322743975953 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 322.9477977811502 < 438.42225884780396.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 43.057743057405254 (T) = (0 -334.0310364949242) / Math.Sqrt((110.05142011503875 / (299)) + (90.84731039005145 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25905001806239697 = (450.8145551490865 - 334.0310364949242) / 450.8145551490865 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 251.91259074335642 < 319.8658296438178.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 35.388832468642626 (T) = (0 -252.89702442604846) / Math.Sqrt((106.75644037599957 / (299)) + (69.00426858736908 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2498473015593708 = (337.1273941315616 - 252.89702442604846) / 337.1273941315616 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 231.15339930331808 < 265.9554478439572.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 50.64596944064838 (T) = (0 -232.37643418090641) / Math.Sqrt((105.17926707314714 / (299)) + (8.00183567078725 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17651719721373874 = (282.1873552121049 - 232.37643418090641) / 282.1873552121049 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 280.0948011974888 < 344.32782664946103.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 61.51782309515719 (T) = (0 -277.52265319903796) / Math.Sqrt((109.29043381155294 / (299)) + (20.938156866490925 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23757857302642954 = (364.00164447195993 - 277.52265319903796) / 364.00164447195993 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 204.51558007282404 < 234.18208669949058.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 30.08275307140541 (T) = (0 -201.47639009170294) / Math.Sqrt((43.518502677205106 / (299)) + (32.79568274351714 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1960762260249897 = (250.61628554097936 - 201.47639009170294) / 250.61628554097936 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToWriter(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 195.92142090087347 < 244.87866898996558.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 42.75991252845789 (T) = (0 -184.939925861789) / Math.Sqrt((116.6515338677854 / (299)) + (40.29900787605079 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.30164638375874686 = (264.822751054388 - 184.939925861789) / 264.822751054388 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeObjectProperty(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 380.35958131724635 < 474.0265811055349.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 85.56024702609317 (T) = (0 -384.72905847440416) / Math.Sqrt((150.46372429339408 / (299)) + (20.911849392941132 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24424565963588765 = (509.06629036235086 - 384.72905847440416) / 509.06629036235086 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 267.63306183415324 < 313.6721953866091.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 45.045869688688775 (T) = (0 -257.6509168611846) / Math.Sqrt((142.3145760393644 / (299)) + (26.39417438773278 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21678135496580078 = (328.96422792633376 - 257.6509168611846) / 328.96422792633376 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Net.Primitives.Tests.CredentialCacheTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ForEach - Duration of single invocation 188.49 ns 131.47 ns 0.70 0.22 False
GetCredential_Uri - Duration of single invocation 863.71 ns 410.30 ns 0.48 0.11 True
ForEach - Duration of single invocation 31.76 ns 18.12 ns 0.57 0.43 False
ForEach - Duration of single invocation 399.20 ns 237.09 ns 0.59 0.16 False
GetCredential_HostPort - Duration of single invocation 59.03 ns 50.75 ns 0.86 0.13 False
GetCredential_Uri - Duration of single invocation 689.08 ns 307.27 ns 0.45 0.13 True
ForEach - Duration of single invocation 204.51 ns 121.40 ns 0.59 0.12 False
GetCredential_Uri - Duration of single invocation 135.13 ns 98.90 ns 0.73 0.06 False
GetCredential_HostPort - Duration of single invocation 26.89 ns 23.25 ns 0.86 0.48 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.CredentialCacheTests*'

Payloads

Baseline
Compare

Histogram

System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 0, hostPortCount: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 131.46963036710176 < 176.6862342247299.
IsChangePoint: Marked as a change because one of 3/3/2023 8:38:01 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 34.766889462675 (T) = (0 -126.83825546453305) / Math.Sqrt((180.2235140503096 / (299)) + (30.552248964374087 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.320200892799855 = (186.58196829197894 - 126.83825546453305) / 186.58196829197894 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString: "http://name5", uriCount: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 410.302759839472 < 815.0333687940497.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 227.20570301655104 (T) = (0 -409.75981497697035) / Math.Sqrt((375.8782597983847 / (299)) + (30.10095002526043 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5117298513803907 = (839.2071809743114 - 409.75981497697035) / 839.2071809743114 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 0, hostPortCount: 0)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.119419442341577 < 30.127667573788617.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 36.27218318291554 (T) = (0 -19.163451334375225) / Math.Sqrt((1.7347738175798693 / (299)) + (1.387376165789294 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3883503301402512 = (31.33076379943016 - 19.163451334375225) / 31.33076379943016 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 10, hostPortCount: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 237.09018128650564 < 361.68313327341343.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 4/6/2023 1:55:21 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 51.599483567244306 (T) = (0 -238.39843999831575) / Math.Sqrt((422.6352359687416 / (299)) + (51.81845724278998 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3346393847610821 = (358.2995965469215 - 238.39843999831575) / 358.2995965469215 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_HostPort(host: "name5", hostPortCount: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 50.75082730915304 < 56.04054245436752.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 19.563581231162896 (T) = (0 -51.21153598625985) / Math.Sqrt((2.527047139896185 / (299)) + (1.673487240177014 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12395265386098146 = (58.4574980016356 - 51.21153598625985) / 58.4574980016356 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString: "http://notfound", uriCount: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 307.26947123795946 < 630.4424154437859.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 159.29519872044972 (T) = (0 -306.8770000130607) / Math.Sqrt((482.3981706108812 / (299)) + (37.77423639630283 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5245967626924446 = (645.5088563364809 - 306.8770000130607) / 645.5088563364809 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 10, hostPortCount: 0)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 121.40415417350816 < 192.92404000473118.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/2/2023 9:04:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 51.95378639129224 (T) = (0 -124.7478608297377) / Math.Sqrt((221.24168376187637 / (299)) + (15.844038122164001 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3682348742003757 = (197.45923878252142 - 124.7478608297377) / 197.45923878252142 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString: "http://notfound", uriCount: 0)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 98.89914005861485 < 127.91801083933528.
IsChangePoint: Marked as a change because one of 3/28/2023 3:45:48 AM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 17.57342513606479 (T) = (0 -102.75329758242559) / Math.Sqrt((22.88238698476749 / (299)) + (28.603903155327515 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20512732413997922 = (129.27013432843265 - 102.75329758242559) / 129.27013432843265 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_HostPort(host: "notfound", hostPortCount: 0)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.253147885820407 < 23.2612279627104.
IsChangePoint: Marked as a change because one of 5/19/2023 8:24:48 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 7.924110965113226 (T) = (0 -23.615912047831827) / Math.Sqrt((2.034745579483858 / (299)) + (0.9219866499365894 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.09174276831739864 = (26.00134766235984 - 23.615912047831827) / 26.00134766235984 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryGetValueTrue<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FrozenDictionary - Duration of single invocation 9.37 μs 8.66 μs 0.92 0.01 False
FrozenDictionaryOptimized - Duration of single invocation 7.64 μs 6.24 μs 0.82 0.00 True
ImmutableDictionary - Duration of single invocation 32.56 μs 24.34 μs 0.75 0.03 True
ImmutableSortedDictionary - Duration of single invocation 221.30 μs 194.13 μs 0.88 0.06 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueTrue<String, String>.FrozenDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.664088060052107 < 9.030632625329764.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 42.20527674469355 (T) = (0 -8703.768628626749) / Math.Sqrt((42163.947050548406 / (265)) + (3864.849692864566 / (13))) is greater than 1.968596344330496 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (265) + (13) - 2, .975) and 0.09386970153612217 = (9605.427214366255 - 8703.768628626749) / 9605.427214366255 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.TryGetValueTrue<String, String>.FrozenDictionaryOptimized(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.23784931039244 < 7.2504306626912145.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 382.06032618910314 (T) = (0 -6246.05006113773) / Math.Sqrt((493.0610874206941 / (264)) + (143.9172043625506 / (13))) is greater than 1.9686278708474185 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (264) + (13) - 2, .975) and 0.18034178781377028 = (7620.310476092201 - 6246.05006113773) / 7620.310476092201 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.TryGetValueTrue<String, String>.ImmutableDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 24.336564842578706 < 30.9722554086713.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 53.58522968502284 (T) = (0 -23700.79985227978) / Math.Sqrt((639703.8952351584 / (299)) + (368319.8312464747 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2829830278200911 = (33054.726417735255 - 23700.79985227978) / 33054.726417735255 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.TryGetValueTrue<String, String>.ImmutableSortedDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 194.13388999118163 < 230.4735122596154.
IsChangePoint: Marked as a change because one of 3/4/2023 1:07:20 AM, 3/14/2023 8:42:09 PM, 3/17/2023 1:30:52 PM, 4/25/2023 6:46:56 PM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.629788480672726 (T) = (0 -201537.07292593046) / Math.Sqrt((107777902.71753486 / (299)) + (90650289.10702495 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.10624062186956017 = (225493.6595434718 - 201537.07292593046) / 225493.6595434718 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractJsonSerializer_ - Duration of single invocation 294.71 μs 231.88 μs 0.79 0.04 True
JsonNet_ - Duration of single invocation 67.72 μs 46.87 μs 0.69 0.07 True
SystemTextJson_SourceGen_ - Duration of single invocation 28.38 μs 20.08 μs 0.71 0.05 True
SystemTextJson_Reflection_ - Duration of single invocation 27.89 μs 20.96 μs 0.75 0.05 True

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.DataContractJsonSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 231.8756729744137 < 286.66231395209655.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 82.85293041866458 (T) = (0 -232010.11838739243) / Math.Sqrt((19344757.291773126 / (299)) + (8700861.239351213 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23427270286664362 = (302993.1403202756 - 232010.11838739243) / 302993.1403202756 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.JsonNet_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 46.87042594324915 < 65.06221018762486.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 41.541045546949064 (T) = (0 -48228.56572649347) / Math.Sqrt((3133074.9933105716 / (299)) + (3352278.9865584634 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.30852837155187024 = (69747.71450093025 - 48228.56572649347) / 69747.71450093025 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.SystemTextJson_SourceGen_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.07886904761905 < 26.979222220441663.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 54.82252644452441 (T) = (0 -20751.3109917157) / Math.Sqrt((284254.7400476199 / (299)) + (208021.88203346787 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2559393354335489 = (27889.272985297597 - 20751.3109917157) / 27889.272985297597 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.SystemTextJson_Reflection_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.95662121055888 < 26.466376641741217.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 63.42850904830986 (T) = (0 -20882.59031154242) / Math.Sqrt((357208.36496657046 / (299)) + (138138.10432045782 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24825242073022966 = (27778.726380239586 - 20882.59031154242) / 27778.726380239586 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.NDhrystone

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 312.22 ms 266.75 ms 0.85 0.10 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.NDhrystone*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.NDhrystone.Test

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 266.746975 < 298.11321903846147.
IsChangePoint: Marked as a change because one of 4/26/2023 6:27:29 AM, 5/1/2023 3:42:23 PM, 5/16/2023 5:40:18 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 16.061413840865924 (T) = (0 -280643539.8074328) / Math.Sqrt((742792915252804.6 / (299)) + (232045367132772.9 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20513225260242954 = (353069476.93659884 - 280643539.8074328) / 353069476.93659884 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Pipelines.Tests.Perf_Pipe

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SyncReadAsyncWithCancellationToken - Duration of single invocation 832.07 μs 731.89 μs 0.88 0.21 False
ReadAsync - Duration of single invocation 2.50 ms 2.22 ms 0.89 0.07 False
SyncReadAsync - Duration of single invocation 851.86 μs 752.74 μs 0.88 0.19 False
ReadAsyncWithCancellationToken - Duration of single invocation 3.10 ms 2.50 ms 0.81 0.07 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Pipelines.Tests.Perf_Pipe*'

Payloads

Baseline
Compare

Histogram

System.IO.Pipelines.Tests.Perf_Pipe.SyncReadAsyncWithCancellationToken

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 731.8896401515151 < 788.8409129464284.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 49.63734320524779 (T) = (0 -732414.5387945388) / Math.Sqrt((569129414.9109696 / (299)) + (14013083.95157578 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1047607932360793 = (818121.6073434107 - 732414.5387945388) / 818121.6073434107 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Pipelines.Tests.Perf_Pipe.ReadAsync

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.222606845238096 < 2.406662679897959.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 36.26290534129649 (T) = (0 -2137955.637762598) / Math.Sqrt((3990183209.6001773 / (299)) + (1612993531.148373 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16585635795046896 = (2563054.5268073236 - 2137955.637762598) / 2563054.5268073236 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Pipelines.Tests.Perf_Pipe.SyncReadAsync

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 752.7417729591835 < 805.9975329427084.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 32.837443339601094 (T) = (0 -740066.7521748703) / Math.Sqrt((357719240.6289459 / (299)) + (74570988.89760776 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10460698749038902 = (826527.2811327938 - 740066.7521748703) / 826527.2811327938 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Pipelines.Tests.Perf_Pipe.ReadAsyncWithCancellationToken

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.5031693571428573 < 2.9055130869997847.
IsChangePoint: Marked as a change because one of 4/18/2023 4:36:42 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 30.856206540648987 (T) = (0 -2547202.7589108227) / Math.Sqrt((10649956608.80036 / (299)) + (3139770494.9853454 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1678205476730443 = (3060881.5824378827 - 2547202.7589108227) / 3060881.5824378827 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsTrueComparer<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedSet - Duration of single invocation 23.47 μs 10.30 μs 0.44 0.01 True
ImmutableHashSet - Duration of single invocation 10.98 μs 6.45 μs 0.59 0.24 True
HashSet - Duration of single invocation 5.38 μs 2.84 μs 0.53 0.01 True
ImmutableSortedSet - Duration of single invocation 22.78 μs 11.63 μs 0.51 0.15 True

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrueComparer&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrueComparer<Int32>.SortedSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.302936032593276 < 22.39059593897526.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 239.46537247483732 (T) = (0 -10361.538520873406) / Math.Sqrt((385594.42796194804 / (299)) + (19331.513276095942 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.549104599787161 = (22979.91621999777 - 10361.538520873406) / 22979.91621999777 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrueComparer<Int32>.ImmutableHashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.449526352951894 < 9.971939789452312.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 54.76281671391365 (T) = (0 -5936.314489200954) / Math.Sqrt((145048.97233917777 / (299)) + (97145.59034652721 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4514348038493139 = (10821.529566323963 - 5936.314489200954) / 10821.529566323963 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrueComparer<Int32>.HashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.8425729473167998 < 5.081086048291403.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 126.69794742691164 (T) = (0 -2904.6613588627815) / Math.Sqrt((11097.5053865116 / (299)) + (3447.1680559199426 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4312910268268073 = (5107.465322123915 - 2904.6613588627815) / 5107.465322123915 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrueComparer<Int32>.ImmutableSortedSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.629823557589951 < 21.251762465851165.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 200.00307676170337 (T) = (0 -11641.504381528684) / Math.Sqrt((754050.5871771855 / (299)) + (1003.8566492749088 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4669150263826841 = (21837.99010978264 - 11641.504381528684) / 21837.99010978264 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractSerializer_BinaryXml_ - Duration of single invocation 662.79 ns 464.52 ns 0.70 0.13 False
DataContractSerializer_ - Duration of single invocation 1.06 μs 744.89 ns 0.70 0.08 True
XmlSerializer_ - Duration of single invocation 1.86 μs 1.53 μs 0.83 0.04 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;LoginViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>.DataContractSerializer_BinaryXml_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 464.51733376935624 < 622.5954288747713.
IsChangePoint: Marked as a change because one of 4/4/2023 8:16:00 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 78.3473435640467 (T) = (0 -458.45659421242334) / Math.Sqrt((1115.7570767912919 / (299)) + (61.28225675439959 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3318370890787183 = (686.1449307030385 - 458.45659421242334) / 686.1449307030385 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>.DataContractSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 744.8850499576093 < 1.002945352891349.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 71.73592284005666 (T) = (0 -743.8707844422331) / Math.Sqrt((537.6202903899319 / (299)) + (219.10259648958058 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2940289195912956 = (1053.6844993871246 - 743.8707844422331) / 1053.6844993871246 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>.XmlSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.5335444979070623 < 1.6957913559910565.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 61.51348204144826 (T) = (0 -1539.4866751336376) / Math.Sqrt((847.7757268319787 / (299)) + (208.28602575411787 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1478586117644139 = (1806.6094387473004 - 1539.4866751336376) / 1806.6094387473004 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Document.Tests.Perf_DocumentParse

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 1.05 ms 674.86 μs 0.64 0.17 True
Parse - Duration of single invocation 231.64 ns 189.53 ns 0.82 0.05 False
Parse - Duration of single invocation 303.87 ns 254.75 ns 0.84 0.13 False
Parse - Duration of single invocation 318.31 ns 265.99 ns 0.84 0.09 False
Parse - Duration of single invocation 1.11 μs 847.61 ns 0.76 0.11 False
Parse - Duration of single invocation 2.82 μs 2.33 μs 0.83 0.06 False
Parse - Duration of single invocation 215.68 ns 168.71 ns 0.78 0.06 False
Parse - Duration of single invocation 1.33 μs 1.01 μs 0.75 0.21 False
Parse - Duration of single invocation 1.09 μs 822.33 ns 0.76 0.15 True
Parse - Duration of single invocation 2.49 μs 2.19 μs 0.88 0.04 False
Parse - Duration of single invocation 721.59 μs 494.32 μs 0.69 0.13 True
Parse - Duration of single invocation 1.61 μs 1.31 μs 0.81 0.03 False
Parse - Duration of single invocation 5.71 ms 5.23 ms 0.92 0.01 False
Parse - Duration of single invocation 1.83 μs 1.52 μs 0.83 0.02 False
Parse - Duration of single invocation 885.92 ns 683.27 ns 0.77 0.07 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 674.8616095494802 < 0.9561960052212573.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 26.924623966111774 (T) = (0 -706478.6547118758) / Math.Sqrt((757109718.8618766 / (299)) + (1777475315.2859924 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3102230432432488 = (1024213.1862938043 - 706478.6547118758) / 1024213.1862938043 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: HelloWorld)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 189.53369031446965 < 219.82586611765302.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 37.21758833602478 (T) = (0 -184.09926093889138) / Math.Sqrt((52.64464691688303 / (299)) + (17.896223980976377 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2012186048088799 = (230.47514882947786 - 184.09926093889138) / 230.47514882947786 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: HelloWorld)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 254.75148324255497 < 288.48954417096314.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 34.7587959493401 (T) = (0 -256.7216271622908) / Math.Sqrt((109.27024522698868 / (299)) + (16.504488464775545 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14757740380770662 = (301.16708345021203 - 256.7216271622908) / 301.16708345021203 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: HelloWorld)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 265.9854612986557 < 302.1770206687276.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 20.47843795079156 (T) = (0 -273.39135402350854) / Math.Sqrt((86.18819474419915 / (299)) + (48.43948449584061 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1304947149714105 = (314.42172776962406 - 273.39135402350854) / 314.42172776962406 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: BasicJson)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 847.6144189577572 < 1.0376667578022238.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 29.780343750067416 (T) = (0 -873.0395648290364) / Math.Sqrt((1097.435604207237 / (299)) + (660.1822066037511 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20109608953445027 = (1092.7967098324568 - 873.0395648290364) / 1092.7967098324568 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: Json400B)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.333465614123175 < 2.6839792573489683.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 47.39395252266628 (T) = (0 -2308.536874440714) / Math.Sqrt((4068.250405909412 / (299)) + (928.8839688581384 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15919857689457234 = (2745.638638329526 - 2308.536874440714) / 2745.638638329526 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: HelloWorld)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 168.7136569064759 < 204.69375138126412.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 27.685874492740297 (T) = (0 -173.0162557388277) / Math.Sqrt((38.077232066675165 / (299)) + (30.213530622012723 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20034811822759366 = (216.3644701933821 - 173.0162557388277) / 216.3644701933821 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400B)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0057384637756093 < 1.2730794861560901.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 44.51721749903407 (T) = (0 -998.3645555151359) / Math.Sqrt((2813.9640680477196 / (299)) + (926.996579179487 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.28602730147355837 = (1398.323153778354 - 998.3645555151359) / 1398.323153778354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: Json400B)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 822.3307694323601 < 1.0399792838899695.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 48.41094448674823 (T) = (0 -810.0368220433492) / Math.Sqrt((894.3833316250215 / (299)) + (440.0093893694862 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2661807532686052 = (1103.8642358475136 - 810.0368220433492) / 1103.8642358475136 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: Json400B)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.1890462845752037 < 2.354067909683226.
IsChangePoint: Marked as a change because one of 4/22/2023 7:07:45 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 37.21634952273442 (T) = (0 -2161.322222538052) / Math.Sqrt((2413.7067461317647 / (299)) + (808.033170906412 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12610485525070358 = (2473.2054360573125 - 2161.322222538052) / 2473.2054360573125 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: Json400KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 494.32272727272726 < 688.617752962843.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 33.307670324273474 (T) = (0 -510561.4849534935) / Math.Sqrt((376090160.10702944 / (299)) + (521018727.0478244 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2954927825975841 = (724707.2454927879 - 510561.4849534935) / 724707.2454927879 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: BasicJson)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3067794157137225 < 1.4710837329089703.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 49.23704045557582 (T) = (0 -1321.921791272795) / Math.Sqrt((934.2213628393049 / (299)) + (236.4479168516915 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14672246576232592 = (1549.2284025194826 - 1321.921791272795) / 1549.2284025194826 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: Json400KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.228457446808512 < 5.419840455627705.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 27.728244438840086 (T) = (0 -5204172.063616962) / Math.Sqrt((25839929425.154854 / (299)) + (547424580.0588158 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.05696425294668409 = (5518531.062982851 - 5204172.063616962) / 5518531.062982851 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: BasicJson)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.5226156989585982 < 1.7227272794808115.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 37.079265064516804 (T) = (0 -1520.228640814445) / Math.Sqrt((1069.6541215725038 / (299)) + (533.5949801253478 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14010344163556168 = (1767.9203690569339 - 1520.228640814445) / 1767.9203690569339 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: BasicJson)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 683.2712423268473 < 840.45493959777.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 14.164488463800426 (T) = (0 -697.1051366459943) / Math.Sqrt((523.2252068307658 / (299)) + (2216.3801247573133 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2105269915065722 = (883.0005955191533 - 697.1051366459943) / 883.0005955191533 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Concurrent.IsEmpty<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 5.43 ns 3.61 ns 0.67 0.02 False
Bag - Duration of single invocation 11.65 ns 8.62 ns 0.74 0.03 True
Bag - Duration of single invocation 11.10 ns 8.11 ns 0.73 0.06 True
Dictionary - Duration of single invocation 4.46 ns 2.87 ns 0.64 0.02 False
Queue - Duration of single invocation 5.78 ns 3.89 ns 0.67 0.02 True

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.IsEmpty&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.IsEmpty<Int32>.Queue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.612293357124291 < 5.154654764159853.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 134.970557244511 (T) = (0 -3.60791396968109) / Math.Sqrt((0.0471157445755119 / (299)) + (2.3573975721814288E-05 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3207890888382297 = (5.311919921177148 - 3.60791396968109) / 5.311919921177148 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Concurrent.IsEmpty<Int32>.Bag(Size: 0)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.619384592220847 < 11.129071966431539.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 48.88300361150407 (T) = (0 -8.652521456452234) / Math.Sqrt((0.047700130012555506 / (299)) + (0.048377508423038194 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26032735579929006 = (11.697771337484228 - 8.652521456452234) / 11.697771337484228 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Concurrent.IsEmpty<Int32>.Bag(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.108700107402166 < 10.550016538713047.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 130.44629852131646 (T) = (0 -8.11808891681153) / Math.Sqrt((0.12473713133381968 / (299)) + (0.001460920678483072 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2699521029631415 = (11.119940143326877 - 8.11808891681153) / 11.119940143326877 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Concurrent.IsEmpty<Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.8675993977592027 < 4.234763664508559.
IsChangePoint: Marked as a change because one of 3/17/2023 9:25:32 PM, 4/19/2023 8:38:27 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 111.42683011824711 (T) = (0 -2.8670689791290362) / Math.Sqrt((0.05418347333537961 / (299)) + (2.4615851121153905E-06 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.34359566730284563 = (4.36783981505469 - 2.8670689791290362) / 4.36783981505469 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Concurrent.IsEmpty<Int32>.Queue(Size: 0)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.886082072380083 < 5.493264625415588.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 182.407315349465 (T) = (0 -3.8920400325969533) / Math.Sqrt((0.02913735364126351 / (299)) + (4.122453457225739E-05 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3197831072137939 = (5.721763269734366 - 3.8920400325969533) / 5.721763269734366 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToString - Duration of single invocation 447.23 ns 383.22 ns 0.86 0.06 False
SerializeToStream - Duration of single invocation 778.98 ns 560.46 ns 0.72 0.06 False
SerializeToStream - Duration of single invocation 453.91 ns 388.40 ns 0.86 0.05 False
SerializeToWriter - Duration of single invocation 629.08 ns 391.37 ns 0.62 0.10 False
SerializeObjectProperty - Duration of single invocation 685.04 ns 537.54 ns 0.78 0.09 False
SerializeObjectProperty - Duration of single invocation 965.41 ns 662.15 ns 0.69 0.08 True
SerializeToWriter - Duration of single invocation 318.46 ns 293.37 ns 0.92 0.05 False
SerializeToString - Duration of single invocation 737.86 ns 483.69 ns 0.66 0.06 False
SerializeToUtf8Bytes - Duration of single invocation 696.05 ns 464.73 ns 0.67 0.06 False
SerializeToUtf8Bytes - Duration of single invocation 389.61 ns 362.39 ns 0.93 0.06 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LargeStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 383.22406759906755 < 420.10078144004126.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 12:29:35 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 45.2761208228055 (T) = (0 -374.47148357460304) / Math.Sqrt((206.2623783005799 / (299)) + (19.665044877295408 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.152137846591713 = (441.66552554478363 - 374.47148357460304) / 441.66552554478363 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 560.4589995308044 < 756.9899644209802.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/10/2023 1:33:42 PM, 5/2/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 62.58471253449321 (T) = (0 -528.0331372127996) / Math.Sqrt((1909.5409079444355 / (299)) + (208.86529943689186 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.35964018185538754 = (824.5881803494951 - 528.0331372127996) / 824.5881803494951 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 388.40486402710445 < 440.19474675814513.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 12:29:35 AM, 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 18.886997914787383 (T) = (0 -400.27741650417494) / Math.Sqrt((257.2618365782893 / (299)) + (175.4185239948196 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1516568384028185 = (471.83431731867785 - 400.27741650417494) / 471.83431731867785 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToWriter(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 391.37144287549523 < 601.5262481900498.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/10/2023 1:33:42 PM, 5/2/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 78.26908482618319 (T) = (0 -394.26244357480203) / Math.Sqrt((909.9180954844783 / (299)) + (96.41442516689106 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3910027230531831 = (647.3960697351896 - 394.26244357480203) / 647.3960697351896 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeObjectProperty(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 537.5404011970923 < 649.6513673712344.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 12:29:35 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 118.03514982737222 (T) = (0 -538.5394045271664) / Math.Sqrt((426.9160396783617 / (299)) + (4.887136285313304 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22741854956015048 = (697.0648909840675 - 538.5394045271664) / 697.0648909840675 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeObjectProperty(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 662.1517215836991 < 926.4377164876049.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 94.60838204340052 (T) = (0 -662.9195492901504) / Math.Sqrt((1385.4654681827506 / (299)) + (90.70137971727944 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.32718507092149285 = (985.2925680440688 - 662.9195492901504) / 985.2925680440688 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToWriter(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 293.37377227622517 < 309.1300087449627.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 12:29:35 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 33.37512137696119 (T) = (0 -285.7235422801565) / Math.Sqrt((115.69504404532277 / (299)) + (15.362850531534173 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12762856641907638 = (327.52510144367534 - 285.7235422801565) / 327.52510144367534 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 483.69098519474176 < 697.574673587519.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 82.7993978569093 (T) = (0 -486.81284488106763) / Math.Sqrt((1247.8662721233675 / (299)) + (73.72802083368995 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.34796765989182565 = (746.6084347906789 - 486.81284488106763) / 746.6084347906789 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 464.7304161973007 < 661.4627867722356.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 83.39851756085672 (T) = (0 -466.54192336155336) / Math.Sqrt((952.3726780122227 / (299)) + (71.44455178475998 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3449859143260428 = (712.2624284965093 - 466.54192336155336) / 712.2624284965093 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 362.39089365149846 < 381.1219707284729.
IsChangePoint: Marked as a change because one of 3/20/2023 4:51:16 PM, 4/12/2023 8:39:35 PM, 4/25/2023 1:57:48 PM, 5/13/2023 6:35:24 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 20.95711984466949 (T) = (0 -358.63184662972554) / Math.Sqrt((185.8469720664312 / (299)) + (58.63729841588316 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11690633611144079 = (406.1085038823046 - 358.63184662972554) / 406.1085038823046 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in LinqBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Order00LinqMethodX - Duration of single invocation 66.48 ms 53.76 ms 0.81 0.00 False
Where01ForX - Duration of single invocation 195.49 ms 163.11 ms 0.83 0.01 False
Where00ForX - Duration of single invocation 208.87 ms 177.74 ms 0.85 0.04 False
Count00LinqMethodX - Duration of single invocation 985.78 ms 745.46 ms 0.76 0.09 False
Order00ManualX - Duration of single invocation 119.01 ms 88.75 ms 0.75 0.06 False
Where01LinqMethodNestedX - Duration of single invocation 342.19 ms 265.42 ms 0.78 0.01 True
Where01LinqMethodX - Duration of single invocation 301.10 ms 251.48 ms 0.84 0.01 False
Where00LinqQueryX - Duration of single invocation 613.70 ms 490.52 ms 0.80 0.02 False
Order00LinqQueryX - Duration of single invocation 67.28 ms 53.94 ms 0.80 0.00 False
Where00LinqMethodX - Duration of single invocation 612.31 ms 480.53 ms 0.78 0.03 False
Where01LinqQueryX - Duration of single invocation 300.21 ms 262.91 ms 0.88 0.01 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'

Payloads

Baseline
Compare

Histogram

LinqBenchmarks.Order00LinqMethodX

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 53.755965384615386 < 64.17438258333333.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 53.179031770796534 (T) = (0 -53858141.722316146) / Math.Sqrt((2322383796632.564 / (299)) + (484242443681.5772 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17320921028252323 = (65141197.013962924 - 53858141.722316146) / 65141197.013962924 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

LinqBenchmarks.Where01ForX

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 163.11319333333336 < 185.62412266666666.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/10/2023 6:18:38 PM, 5/11/2023 5:02:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 32.80397818286568 (T) = (0 -162033614.94505495) / Math.Sqrt((46365584309540.46 / (299)) + (6568274166097.016 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14127154760220942 = (188690166.83052185 - 162033614.94505495) / 188690166.83052185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

LinqBenchmarks.Where00ForX

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 177.7371642857143 < 208.28846592032968.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/10/2023 12:53:36 AM, 3/14/2023 8:42:09 PM, 5/1/2023 3:42:23 PM, 5/9/2023 1:22:51 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 35.177007747591624 (T) = (0 -177820064.96196112) / Math.Sqrt((100738449886979.67 / (299)) + (243849522649.6997 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1055289079672355 = (198799118.8824776 - 177820064.96196112) / 198799118.8824776 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

LinqBenchmarks.Count00LinqMethodX

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 745.45819 < 941.8679394505494.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 36.83854066392693 (T) = (0 -739530716.5741522) / Math.Sqrt((1155068509285185.5 / (299)) + (526256786128497.7 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24908878143378496 = (984844410.7496587 - 739530716.5741522) / 984844410.7496587 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

LinqBenchmarks.Order00ManualX

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 88.746035 < 110.16626074999999.
IsChangePoint: Marked as a change because one of 5/1/2023 10:17:01 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 47.93885559914204 (T) = (0 -90624807.185827) / Math.Sqrt((20700038320557.547 / (299)) + (990192515902.6963 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16785076337546154 = (108904512.793198 - 90624807.185827) / 108904512.793198 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

LinqBenchmarks.Where01LinqMethodNestedX

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 265.4193785714286 < 325.4513831666667.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 57.87403639576937 (T) = (0 -269766829.74358976) / Math.Sqrt((10701764378033.635 / (299)) + (21784106907220.777 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2191534872046455 = (345479969.90836364 - 269766829.74358976) / 345479969.90836364 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

LinqBenchmarks.Where01LinqMethodX

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 251.47935333333334 < 284.80001233333326.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 25.380434974826514 (T) = (0 -256347570.4029304) / Math.Sqrt((23242480062525.547 / (299)) + (25731658875468.72 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12434536667364014 = (292749630.55827135 - 256347570.4029304) / 292749630.55827135 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

LinqBenchmarks.Where00LinqQueryX

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 490.52189 < 567.4479641071429.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 73.32081183632519 (T) = (0 -492270976.6329285) / Math.Sqrt((208005511400436.47 / (299)) + (32368508741572.5 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21000895612760231 = (623134882.9220929 - 492270976.6329285) / 623134882.9220929 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

LinqBenchmarks.Order00LinqQueryX

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 53.94105294117647 < 63.877645759615376.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 44.31169853185022 (T) = (0 -54209154.43339685) / Math.Sqrt((2260711134815.3247 / (299)) + (705222637225.8165 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16889824170474305 = (65225652.44548373 - 54209154.43339685) / 65225652.44548373 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

LinqBenchmarks.Where00LinqMethodX

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 480.53254 < 567.6692246666667.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 43.536242677838224 (T) = (0 -494810125.7004197) / Math.Sqrt((224196922225912.12 / (299)) + (103342813637236.36 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20604031868132394 = (623218202.8167939 - 494810125.7004197) / 623218202.8167939 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

LinqBenchmarks.Where01LinqQueryX

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 262.9121466666667 < 284.3520936666667.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 28.175353578084085 (T) = (0 -257723528.216399) / Math.Sqrt((21804975488438.44 / (299)) + (19034979004649.24 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11936334132848628 = (292655916.23813206 - 257723528.216399) / 292655916.23813206 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.DependencyInjection.TimeToFirstService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BuildProvider - Duration of single invocation 1.08 μs 992.13 ns 0.92 0.02 False
BuildProvider - Duration of single invocation 1.09 μs 993.80 ns 0.92 0.02 False
Singleton - Duration of single invocation 2.58 μs 2.17 μs 0.84 0.01 True
Scoped - Duration of single invocation 2.81 μs 2.26 μs 0.80 0.01 True
Scoped - Duration of single invocation 2.74 μs 2.33 μs 0.85 0.01 True
Transient - Duration of single invocation 2.55 μs 2.07 μs 0.81 0.03 True
Singleton - Duration of single invocation 2.53 μs 2.16 μs 0.85 0.01 True
BuildProvider - Duration of single invocation 1.08 μs 988.39 ns 0.91 0.03 False
Transient - Duration of single invocation 2.50 μs 2.09 μs 0.84 0.02 True
Transient - Duration of single invocation 2.53 μs 2.13 μs 0.84 0.01 True
Scoped - Duration of single invocation 2.75 μs 2.26 μs 0.82 0.02 True
Transient - Duration of single invocation 2.53 μs 2.05 μs 0.81 0.01 True
Scoped - Duration of single invocation 2.77 μs 2.26 μs 0.82 0.02 True
Singleton - Duration of single invocation 2.56 μs 2.18 μs 0.85 0.01 True
Singleton - Duration of single invocation 2.53 μs 2.19 μs 0.86 0.01 True

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.TimeToFirstService*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Expressions")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 992.131826771988 < 1.0135657571343184.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 22.887068879941474 (T) = (0 -991.0576989545687) / Math.Sqrt((1413.103143030167 / (299)) + (400.47603998502433 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1210010390489656 = (1127.4844942732266 - 991.0576989545687) / 1127.4844942732266 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Dynamic")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 993.7999453087571 < 1.018517412881989.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 38.06150247229377 (T) = (0 -983.8766923580735) / Math.Sqrt((1493.3301696104736 / (299)) + (120.55492706270036 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12749490189019358 = (1127.6457805112454 - 983.8766923580735) / 1127.6457805112454 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Dynamic")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.1712138461386488 < 2.423898787502506.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 51.94897178136494 (T) = (0 -2158.7425291572517) / Math.Sqrt((1319.901327725644 / (299)) + (723.1013548162869 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15715718434942363 = (2561.2634871793434 - 2158.7425291572517) / 2561.2634871793434 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Dynamic")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.258089208186892 < 2.6084158533967536.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 49.405427832346675 (T) = (0 -2276.850573817267) / Math.Sqrt((3280.4006206905783 / (299)) + (1575.699654951173 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1996617170241886 = (2844.860257529478 - 2276.850573817267) / 2844.860257529478 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "ILEmit")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.330031898565996 < 2.6165747634224985.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 63.68494069795486 (T) = (0 -2284.853019223957) / Math.Sqrt((3215.6423739797306 / (299)) + (859.5499590480941 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19638776740041322 = (2843.228271715997 - 2284.853019223957) / 2843.228271715997 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "ILEmit")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.0740337355922502 < 2.3651684829820128.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 59.23131432729719 (T) = (0 -2084.766085429313) / Math.Sqrt((1424.5121840864845 / (299)) + (621.3264491820879 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17079587322110373 = (2514.177170738089 - 2084.766085429313) / 2514.177170738089 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "ILEmit")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.1635381858625795 < 2.4112694611551886.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 64.2219374381786 (T) = (0 -2148.2201961628825) / Math.Sqrt((1428.8286007122854 / (299)) + (463.09919206445744 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15967957817030468 = (2556.4298336167944 - 2148.2201961628825) / 2556.4298336167944 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Runtime")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 988.3902885306933 < 1.0188230613839069.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 43.39263310450158 (T) = (0 -983.3364524820392) / Math.Sqrt((1418.1293404576322 / (299)) + (82.57792740792136 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12815045569925806 = (1127.8740224274754 - 983.3364524820392) / 1127.8740224274754 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Runtime")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.09368275880336 < 2.3670487519139414.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 81.70696573611285 (T) = (0 -2100.6747707304585) / Math.Sqrt((1646.8410833478367 / (299)) + (263.3582734796309 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1648819041101783 = (2515.422406806047 - 2100.6747707304585) / 2515.422406806047 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Dynamic")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.1270338656495102 < 2.3659973795351195.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 55.334345353247684 (T) = (0 -2087.7612304182667) / Math.Sqrt((1172.7307379301567 / (299)) + (712.9352806363245 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16886472678047484 = (2511.93914840242 - 2087.7612304182667) / 2511.93914840242 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Runtime")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.262805943604675 < 2.6383768861515433.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 123.18367318515885 (T) = (0 -2264.054986563689) / Math.Sqrt((3653.4684578981633 / (299)) + (131.5947644289134 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20456387428394318 = (2846.3064643004136 - 2264.054986563689) / 2846.3064643004136 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Expressions")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.0541095224384236 < 2.383861683199384.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 50.74406193056924 (T) = (0 -2098.867314241382) / Math.Sqrt((1512.6700571494505 / (299)) + (816.7785166431604 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16611332439692145 = (2516.9694823621585 - 2098.867314241382) / 2516.9694823621585 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Expressions")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.2619128423524058 < 2.6140211335194423.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 75.56692180215305 (T) = (0 -2271.492723868783) / Math.Sqrt((3348.5335172483055 / (299)) + (603.6482037185723 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20163269173994902 = (2845.17251691986 - 2271.492723868783) / 2845.17251691986 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Runtime")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.182527555380515 < 2.4152170581794157.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 51.598858132359844 (T) = (0 -2155.421682618865) / Math.Sqrt((1278.5837392835977 / (299)) + (729.3509809146577 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15684243479211246 = (2556.368787472633 - 2155.421682618865) / 2556.368787472633 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Expressions")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.1866682317717157 < 2.4154278942126037.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 53.08421103774679 (T) = (0 -2153.9347326485117) / Math.Sqrt((1430.1420272753724 / (299)) + (690.1925338460745 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15788750278155403 = (2557.775522585287 - 2153.9347326485117) / 2557.775522585287 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in V8.Crypto.Support

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Bench - Duration of single invocation 4.56 ms 2.85 ms 0.62 0.00 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'V8.Crypto.Support*'

Payloads

Baseline
Compare

Histogram

V8.Crypto.Support.Bench

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.849694660633484 < 4.3416570578703695.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 412.07686861407194 (T) = (0 -2855000.161702551) / Math.Sqrt((4533028302.876131 / (299)) + (44398195.279137544 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.38350787129471997 = (4631040.7363974815 - 2855000.161702551) / 4631040.7363974815 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromStream - Duration of single invocation 1.71 μs 1.22 μs 0.71 0.03 True
DeserializeFromUtf8Bytes - Duration of single invocation 1.30 μs 952.74 ns 0.74 0.13 False
DeserializeFromReader - Duration of single invocation 1.88 μs 1.33 μs 0.71 0.05 True
DeserializeFromStream - Duration of single invocation 1.75 μs 1.26 μs 0.72 0.07 True
DeserializeFromString - Duration of single invocation 1.36 μs 989.20 ns 0.73 0.07 False
DeserializeFromReader - Duration of single invocation 1.91 μs 1.35 μs 0.71 0.06 True
DeserializeFromUtf8Bytes - Duration of single invocation 1.27 μs 899.63 ns 0.71 0.11 True
DeserializeFromString - Duration of single invocation 1.42 μs 1.00 μs 0.71 0.06 True

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2173892671416118 < 1.6404847904837263.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 100.89600012038126 (T) = (0 -1206.9081135668569) / Math.Sqrt((1492.5548519355364 / (299)) + (267.7596717208955 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.29720335321749847 = (1717.2935003208197 - 1206.9081135668569) / 1717.2935003208197 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 952.7445344005721 < 1.2171391506187685.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 70.05623136657759 (T) = (0 -946.1991017223754) / Math.Sqrt((2073.774494521242 / (299)) + (205.566542699818 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26097542501957754 = (1280.335098122334 - 946.1991017223754) / 1280.335098122334 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromReader(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3342984764988421 < 1.7946746784801628.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 91.62278888482972 (T) = (0 -1343.7391672769054) / Math.Sqrt((2084.6451615253986 / (299)) + (335.57282214928586 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2807911777426312 = (1868.357458490753 - 1343.7391672769054) / 1868.357458490753 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2562247330696064 < 1.6617639953755186.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 105.39812400865603 (T) = (0 -1226.1658484086684) / Math.Sqrt((1274.7869479710564 / (299)) + (240.51876470272077 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2908433540943786 = (1729.0479550435653 - 1226.1658484086684) / 1729.0479550435653 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 989.1992009802483 < 1.3192098404050159.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 63.15584797555009 (T) = (0 -1020.4242771883705) / Math.Sqrt((2073.1713046219465 / (299)) + (282.6360151920587 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24892441891775458 = (1358.6172988316478 - 1020.4242771883705) / 1358.6172988316478 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromReader(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.353388518748063 < 1.814895444771594.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 96.9130057260101 (T) = (0 -1364.2616989808857) / Math.Sqrt((2527.5196112513477 / (299)) + (266.93348515436395 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27665033897606806 = (1886.0335083999566 - 1364.2616989808857) / 1886.0335083999566 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 899.6331059251141 < 1.2364380217251123.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 63.47311614305533 (T) = (0 -912.0450851275677) / Math.Sqrt((1409.3352467703135 / (299)) + (336.29976294300724 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2779093718687401 = (1263.0617952872508 - 912.0450851275677) / 1263.0617952872508 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.003896365789802 < 1.3217072742307818.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 45.335608298530936 (T) = (0 -1009.6424492906355) / Math.Sqrt((2088.2097200045787 / (299)) + (696.6218097107021 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2589646716126963 = (1362.4754591496935 - 1009.6424492906355) / 1362.4754591496935 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in GuardedDevirtualization.TwoClassVirtual

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Call - Duration of single invocation 6.22 ns 1.67 ns 0.27 0.01 True
Call - Duration of single invocation 5.91 ns 1.26 ns 0.21 0.02 True
Call - Duration of single invocation 5.53 ns 1.19 ns 0.21 0.04 True
Call - Duration of single invocation 6.47 ns 1.74 ns 0.27 0.02 True
Call - Duration of single invocation 4.62 ns 0.63 ns 0.14 0.05 True
Call - Duration of single invocation 6.52 ns 1.70 ns 0.26 0.03 True
Call - Duration of single invocation 1.68 ns 0.49 ns 0.29 0.04 False
Call - Duration of single invocation 6.25 ns 1.50 ns 0.24 0.02 True
Call - Duration of single invocation 4.33 ns 0.59 ns 0.14 0.05 True
Call - Duration of single invocation 5.34 ns 1.13 ns 0.21 0.03 True

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'GuardedDevirtualization.TwoClassVirtual*'

Payloads

Baseline
Compare

Histogram

GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.60)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.6701475634537228 < 5.912063482322544.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 123.1541798138135 (T) = (0 -1.7249233261935184) / Math.Sqrt((0.003461353353712072 / (299)) + (0.01722813696625777 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7230262531486559 = (6.227750268040063 - 1.7249233261935184) / 6.227750268040063 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.70)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.263574231523872 < 5.615562547443173.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 44.93856980567563 (T) = (0 -1.4140503391116472) / Math.Sqrt((0.0011103298709215011 / (299)) + (0.1313527385248236 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7616256575754258 = (5.932057639798531 - 1.4140503391116472) / 5.932057639798531 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.20)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1860291001286614 < 5.249491822396284.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 202.13822540808144 (T) = (0 -1.167060687916084) / Math.Sqrt((0.0227638289501344 / (299)) + (0.0053714138037266835 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7930182006462031 = (5.638470104906234 - 1.167060687916084) / 5.638470104906234 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.50)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.7374795885863734 < 6.146778017985362.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 149.54103759488083 (T) = (0 -1.910441772870961) / Math.Sqrt((0.0014350711196601375 / (299)) + (0.012071283756385406 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.705136004024359 = (6.479060851596085 - 1.910441772870961) / 6.479060851596085 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.6304492195362505 < 4.384248998745432.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 117.44120861801177 (T) = (0 -0.7774301847976357) / Math.Sqrt((0.014728448422708632 / (299)) + (0.012712863884825435 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.8288113169484828 = (4.541364364393627 - 0.7774301847976357) / 4.541364364393627 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.40)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.6962283394287705 < 6.198786790056721.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 98.27346713541232 (T) = (0 -1.8492146084114682) / Math.Sqrt((0.006132952284116245 / (299)) + (0.02896284515707132 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.715903191998185 = (6.509100265567411 - 1.8492146084114682) / 6.509100265567411 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 1.00)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.49430189558896764 < 1.5965913268624226.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/9/2023 11:04:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 129.82668849096564 (T) = (0 -0.49480447230314717) / Math.Sqrt((0.0188393592342805 / (299)) + (1.2375463960212503E-06 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6757750208191587 = (1.5261145934939289 - 0.49480447230314717) / 1.5261145934939289 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.30)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.5038268270414583 < 5.935478868859124.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 93.01860556324341 (T) = (0 -1.5439969499063375) / Math.Sqrt((0.0018410083152861363 / (299)) + (0.033625657130077716 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7541567749008926 = (6.280412849627652 - 1.5439969499063375) / 6.280412849627652 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.90)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.590622084661432 < 4.120827177372685.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 116.35423058640608 (T) = (0 -0.6750662164092109) / Math.Sqrt((0.028557547684861108 / (299)) + (0.010357364328667006 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.8373567065914344 = (4.150593622778043 - 0.6750662164092109) / 4.150593622778043 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.80)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1300456062024435 < 5.0742436463320795.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 35.744091560278775 (T) = (0 -1.170366232419726) / Math.Sqrt((0.008877825189456975 / (299)) + (0.18198153798157238 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7834232263243508 = (5.403932344899068 - 1.170366232419726) / 5.403932344899068 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<BinaryData>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeObjectProperty - Duration of single invocation 721.96 ns 607.68 ns 0.84 0.04 True
SerializeToString - Duration of single invocation 475.68 ns 439.32 ns 0.92 0.03 False
SerializeToStream - Duration of single invocation 365.34 ns 328.62 ns 0.90 0.05 False
SerializeToString - Duration of single invocation 541.34 ns 479.48 ns 0.89 0.05 False
SerializeToUtf8Bytes - Duration of single invocation 335.10 ns 299.47 ns 0.89 0.04 False
SerializeToStream - Duration of single invocation 464.18 ns 386.90 ns 0.83 0.12 False
SerializeObjectProperty - Duration of single invocation 733.17 ns 647.62 ns 0.88 0.04 False
SerializeToWriter - Duration of single invocation 204.61 ns 174.68 ns 0.85 0.11 False
SerializeToWriter - Duration of single invocation 272.15 ns 209.86 ns 0.77 0.10 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;BinaryData&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeObjectProperty(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 607.675290873545 < 684.6517634919437.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 79.24047029558903 (T) = (0 -606.6680896372745) / Math.Sqrt((174.74692098147898 / (299)) + (22.362048338831702 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1654750642639868 = (726.9622076687508 - 606.6680896372745) / 726.9622076687508 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeToString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 439.317060091708 < 453.1613482354985.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 36.610682153802664 (T) = (0 -439.85549721552445) / Math.Sqrt((58.1778710384373 / (299)) + (15.250678355663645 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08870588052352757 = (482.6712779274996 - 439.85549721552445) / 482.6712779274996 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeToStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 328.6162836073079 < 350.30056339677816.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 30.644446781073782 (T) = (0 -328.1936313751294) / Math.Sqrt((145.30131996240442 / (299)) + (26.60406261081721 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12936737847474925 = (376.95995217841823 - 328.1936313751294) / 376.95995217841823 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeToString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 479.47871826669683 < 514.1774799452301.
IsChangePoint: Marked as a change because one of 3/20/2023 10:25:17 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 26.751716229945945 (T) = (0 -483.9797168315605) / Math.Sqrt((121.40079082913667 / (299)) + (50.60817320732014 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10282172674832889 = (539.4465417418745 - 483.9797168315605) / 539.4465417418745 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeToUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 299.47346507460367 < 322.80252559008676.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 52.73864743691894 (T) = (0 -298.41443939434237) / Math.Sqrt((48.39019871047425 / (299)) + (7.10465074490682 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12948251610724273 = (342.8012014875342 - 298.41443939434237) / 342.8012014875342 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeToStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 386.9007472054999 < 444.7875550782334.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 26.78591807018241 (T) = (0 -385.684035511918) / Math.Sqrt((193.44089632155135 / (299)) + (93.45773331925966 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1627677868287398 = (460.66554707806534 - 385.684035511918) / 460.66554707806534 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeObjectProperty(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 647.6248024768208 < 704.7915627167422.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 57.12298160663685 (T) = (0 -636.172871011447) / Math.Sqrt((153.23402361835855 / (299)) + (39.183136982237244 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14429077052375572 = (743.4451436276206 - 636.172871011447) / 743.4451436276206 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeToWriter(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 174.68027756253113 < 193.81567020371753.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 21.992782390555995 (T) = (0 -177.83171006001356) / Math.Sqrt((62.94010146998873 / (299)) + (10.591812871945882 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11128800419644175 = (200.10049476064646 - 177.83171006001356) / 200.10049476064646 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeToWriter(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 209.85766739739208 < 254.44014568831565.
IsChangePoint: Marked as a change because one of 3/20/2023 4:51:16 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 29.67293281920401 (T) = (0 -217.16472989207142) / Math.Sqrt((120.27803373085159 / (299)) + (28.666055823654037 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18075303435176643 = (265.07846717532686 - 217.16472989207142) / 265.07846717532686 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Get - Duration of single invocation 6.53 ms 4.72 ms 0.72 0.13 False
Get - Duration of single invocation 14.15 ms 12.59 ms 0.89 0.10 False
Get - Duration of single invocation 3.16 ms 2.40 ms 0.76 0.18 False
Get - Duration of single invocation 873.99 μs 647.12 μs 0.74 0.12 False
Get - Duration of single invocation 1.43 ms 1.04 ms 0.73 0.21 False
Get - Duration of single invocation 1.91 ms 1.46 ms 0.77 0.19 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 32, KeysCountPerProvider: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.715912587412587 < 7.190544102083333.
IsChangePoint: Marked as a change because one of 4/18/2023 7:42:00 PM, 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 91.86560845444376 (T) = (0 -4723088.0190303335) / Math.Sqrt((123930345995.5602 / (299)) + (2445078835.4267163 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3231587687674514 = (6978132.833943118 - 4723088.0190303335) / 6978132.833943118 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 32, KeysCountPerProvider: 20)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.59450875 < 15.139826999999999.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 17.539019555910667 (T) = (0 -11918524.070512822) / Math.Sqrt((450441652543.11163 / (299)) + (421542866944.94086 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21326577416444073 = (15149365.159313655 - 11918524.070512822) / 15149365.159313655 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 16, KeysCountPerProvider: 20)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.396028012820513 < 3.4524117690172327.
IsChangePoint: Marked as a change because one of 4/19/2023 10:34:24 AM, 5/13/2023 1:49:08 AM, 5/19/2023 8:24:48 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 20.36331195460906 (T) = (0 -2459842.972087096) / Math.Sqrt((27602174330.561745 / (299)) + (25908324626.08654 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.29055649876841894 = (3467285.2282343176 - 2459842.972087096) / 3467285.2282343176 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 8, KeysCountPerProvider: 20)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 647.117800865801 < 1.0384520386545608.
IsChangePoint: Marked as a change because one of 4/18/2023 7:36:21 AM, 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 28.046719527068102 (T) = (0 -661521.2506262914) / Math.Sqrt((2330088290.6607404 / (299)) + (1287501164.0079677 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3046939932295419 = (951410.2340333727 - 661521.2506262914) / 951410.2340333727 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 16, KeysCountPerProvider: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.037307579529737 < 1.560644958333333.
IsChangePoint: Marked as a change because one of 4/19/2023 4:18:46 AM, 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 38.1737142756598 (T) = (0 -1049962.3651937088) / Math.Sqrt((8989458627.31109 / (299)) + (1746719618.749712 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3179680383776008 = (1539462.1136172074 - 1049962.3651937088) / 1539462.1136172074 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Configuration.ConfigurationBinderBenchmarks.Get(ConfigurationProvidersCount: 8, KeysCountPerProvider: 40)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.4591197674418608 < 2.0388338283984537.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 13.671879762902043 (T) = (0 -1523737.7348890714) / Math.Sqrt((7850880191.489881 / (299)) + (22776999280.600163 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2745090522527334 = (2100284.9720185394 - 1523737.7348890714) / 2100284.9720185394 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IterateTryGetMemory - Duration of single invocation 20.37 ns 17.56 ns 0.86 0.05 False
IterateGetPositionTenSegments - Duration of single invocation 76.48 ns 61.91 ns 0.81 0.04 False
IterateGetPositionMemory - Duration of single invocation 34.47 ns 30.83 ns 0.89 0.06 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateTryGetMemory

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.558190834536124 < 19.365324457631917.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 23.5728789516849 (T) = (0 -17.653889416008393) / Math.Sqrt((0.23086284677396887 / (299)) + (0.1588783486762451 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1321006964050915 = (20.340942022749147 - 17.653889416008393) / 20.340942022749147 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateGetPositionTenSegments

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 61.90774188071358 < 65.59879692550923.
IsChangePoint: Marked as a change because one of 4/26/2023 6:27:29 AM, 5/13/2023 2:42:33 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 22.386786809269438 (T) = (0 -65.81859922024128) / Math.Sqrt((18.128583131892906 / (299)) + (10.914755012382303 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.1719006603245775 = (79.48152602807194 - 65.81859922024128) / 79.48152602807194 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateGetPositionMemory

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 30.826206252012067 < 32.758550283557284.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.903088802090561 (T) = (0 -31.77346960382966) / Math.Sqrt((0.5958459453211815 / (299)) + (0.6402307903959955 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0841871706957779 = (34.69428314076925 - 31.77346960382966) / 34.69428314076925 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadAsyncThenWriteAsync - Duration of single invocation 84.34 ns 63.41 ns 0.75 0.10 False
PingPong - Duration of single invocation 16.18 ms 14.21 ms 0.88 0.13 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests*'

Payloads

Baseline
Compare

Histogram

System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests.ReadAsyncThenWriteAsync

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 63.40659976041544 < 78.47964308254488.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 101.1649600896302 (T) = (0 -63.28716320080479) / Math.Sqrt((14.660474167194632 / (299)) + (0.04931994997502611 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2686838555027937 = (86.53872019236758 - 63.28716320080479) / 86.53872019236758 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests.PingPong

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.208194642857144 < 15.92333702662722.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 39.807366060582765 (T) = (0 -14289967.415832032) / Math.Sqrt((198304521507.73572 / (299)) + (37583401848.26512 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14242288663361033 = (16663186.543933352 - 14289967.415832032) / 16663186.543933352 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromStream - Duration of single invocation 310.21 ns 260.37 ns 0.84 0.06 False
DeserializeFromString - Duration of single invocation 140.69 ns 108.12 ns 0.77 0.07 False
DeserializeFromUtf8Bytes - Duration of single invocation 86.25 ns 67.27 ns 0.78 0.05 False
DeserializeFromStream - Duration of single invocation 311.47 ns 258.46 ns 0.83 0.06 False
DeserializeFromUtf8Bytes - Duration of single invocation 85.61 ns 69.79 ns 0.82 0.05 False
DeserializeFromReader - Duration of single invocation 191.76 ns 154.96 ns 0.81 0.03 True
DeserializeFromString - Duration of single invocation 139.76 ns 107.74 ns 0.77 0.05 False
DeserializeFromReader - Duration of single invocation 193.10 ns 154.69 ns 0.80 0.03 True

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 260.3703321231095 < 303.39069460027304.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 53.85256422990121 (T) = (0 -262.1463469286678) / Math.Sqrt((68.30856103477937 / (299)) + (9.89916556487866 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16970591627821155 = (315.7271044900119 - 262.1463469286678) / 315.7271044900119 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 108.1176302008097 < 133.34887832956318.
IsChangePoint: Marked as a change because one of 3/20/2023 4:51:16 PM, 4/5/2023 11:16:13 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 58.31237031606687 (T) = (0 -108.52936119040031) / Math.Sqrt((29.499492228580714 / (299)) + (2.4705033967172003 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22401999912422257 = (139.86102872227787 - 108.52936119040031) / 139.86102872227787 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 67.26621981186351 < 80.79281883120137.
IsChangePoint: Marked as a change because one of 3/28/2023 3:45:48 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 56.98921688649198 (T) = (0 -69.45682668467666) / Math.Sqrt((8.022387467977726 / (299)) + (0.6299942959812609 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18376641048054582 = (85.09430091644276 - 69.45682668467666) / 85.09430091644276 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 258.4639968161265 < 303.32017414060033.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 49.966739231166635 (T) = (0 -260.2984938331157) / Math.Sqrt((79.43112572459387 / (299)) + (12.391360433027517 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17486645396285774 = (315.4622607252459 - 260.2984938331157) / 315.4622607252459 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 69.7920782999202 < 80.0153196887987.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 69.6208361580293 (T) = (0 -69.10727262234036) / Math.Sqrt((6.263280274114026 / (299)) + (0.3292499688412271 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17811371470955756 = (84.08373987883114 - 69.10727262234036) / 84.08373987883114 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromReader(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 154.96400371768303 < 181.97372927148928.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 101.08124546189454 (T) = (0 -155.13353098549484) / Math.Sqrt((12.912267844049163 / (299)) + (1.1821971302371275 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19265340483958945 = (192.15233199153033 - 155.13353098549484) / 192.15233199153033 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 107.73721090295501 < 132.35375727447527.
IsChangePoint: Marked as a change because one of 3/20/2023 4:51:16 PM, 4/17/2023 6:40:18 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 22.650510306263676 (T) = (0 -112.23648688957809) / Math.Sqrt((14.433261888443404 / (299)) + (19.77769253799927 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20181243000168816 = (140.6141752994418 - 112.23648688957809) / 140.6141752994418 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromReader(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 154.69012432909523 < 183.32996470026.
IsChangePoint: Marked as a change because one of 3/18/2023 3:40:27 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 52.9809617958914 (T) = (0 -156.39958187921383) / Math.Sqrt((26.973495589326188 / (299)) + (4.944228007537178 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18855586245571015 = (192.7422660942908 - 156.39958187921383) / 192.7422660942908 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryAddGiventSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentDictionary - Duration of single invocation 33.05 μs 28.25 μs 0.85 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddGiventSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryAddGiventSize<String>.ConcurrentDictionary(Count: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 28.252671678635547 < 31.149986233579583.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 74.86106990570651 (T) = (0 -27763.103336085005) / Math.Sqrt((422005.010185542 / (299)) + (48293.7406420844 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16181836041967138 = (33123.015376459196 - 27763.103336085005) / 33123.015376459196 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Utf8JsonReaderCommentParsing - Duration of single invocation 103.38 ns 83.85 ns 0.81 0.01 True
Utf8JsonReaderCommentParsing - Duration of single invocation 115.63 ns 96.97 ns 0.84 0.06 False
Utf8JsonReaderCommentParsing - Duration of single invocation 11.09 μs 9.24 μs 0.83 0.33 False
Utf8JsonReaderCommentParsing - Duration of single invocation 103.92 ns 85.88 ns 0.83 0.02 True
Utf8JsonReaderCommentParsing - Duration of single invocation 11.31 μs 9.32 μs 0.82 0.01 True
Utf8JsonReaderCommentParsing - Duration of single invocation 116.21 ns 104.47 ns 0.90 0.04 False
Utf8JsonReaderCommentParsing - Duration of single invocation 102.89 ns 88.66 ns 0.86 0.05 False
Utf8JsonReaderCommentParsing - Duration of single invocation 134.39 ns 116.72 ns 0.87 0.01 False
Utf8JsonReaderCommentParsing - Duration of single invocation 113.39 ns 98.15 ns 0.87 0.04 False
Utf8JsonReaderCommentParsing - Duration of single invocation 114.80 ns 103.38 ns 0.90 0.07 False
Utf8JsonReaderCommentParsing - Duration of single invocation 106.19 ns 92.57 ns 0.87 0.07 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: ShortMultiLine)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 83.84619144170564 < 97.897030266206.
IsChangePoint: Marked as a change because one of 4/25/2023 1:57:48 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 125.95207186414913 (T) = (0 -84.55433120439277) / Math.Sqrt((7.522962833055304 / (299)) + (0.12563808995108117 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21751542179211714 = (108.05878295780188 - 84.55433120439277) / 108.05878295780188 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: ShortMultiLine)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 96.97166118802282 < 112.15667721588638.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 62.570314342597285 (T) = (0 -96.0629188136891) / Math.Sqrt((40.872396360771205 / (299)) + (0.5666611954725465 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2166465128979894 = (122.63035831891241 - 96.0629188136891) / 122.63035831891241 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: LongMultiLine)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.241743230415667 < 10.480106146804584.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 65.93391165712606 (T) = (0 -8978.123132108029) / Math.Sqrt((227798.06237524003 / (299)) + (9896.145207139702 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22276263939873625 = (11551.327287152839 - 8978.123132108029) / 11551.327287152839 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: ShortSingleLine)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 85.8816868327402 < 98.95861638195689.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 142.49760596915257 (T) = (0 -85.7981205810535) / Math.Sqrt((3.020436402194525 / (299)) + (0.10824670047366845 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1839815702835714 = (105.14238092743673 - 85.7981205810535) / 105.14238092743673 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: LongMultiLine)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.318766532418456 < 10.712796627457532.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 82.67099892251011 (T) = (0 -9324.14307988263) / Math.Sqrt((57159.50717001328 / (299)) + (6042.515178790679 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18506065898042315 = (11441.517927232624 - 9324.14307988263) / 11441.517927232624 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: ShortMultiLine)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 104.47263132541991 < 112.69757817524788.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 20.467315729001886 (T) = (0 -104.50859340318335) / Math.Sqrt((27.477395979350938 / (299)) + (5.952305045927719 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12679816393640914 = (119.6843491240352 - 104.50859340318335) / 119.6843491240352 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: ShortMultiLine)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 88.65749595982372 < 97.50139171471834.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/5/2023 12:43:09 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 57.067798510355864 (T) = (0 -87.37356965808478) / Math.Sqrt((11.691098328034187 / (299)) + (1.167188770049666 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18994425001259801 = (107.86118073903384 - 87.37356965808478) / 107.86118073903384 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongSingleLine)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 116.72276454660144 < 128.71146963127103.
IsChangePoint: Marked as a change because one of 4/25/2023 9:59:52 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 83.16141676485455 (T) = (0 -117.75484905197445) / Math.Sqrt((5.583794473129574 / (299)) + (0.49808927777399753 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14427030348143696 = (137.6075290258669 - 117.75484905197445) / 137.6075290258669 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: ShortSingleLine)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 98.14521731613928 < 107.68501943936076.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/19/2023 8:38:27 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 59.495591623219674 (T) = (0 -97.97092199705996) / Math.Sqrt((15.521276344247493 / (299)) + (0.22142345537369224 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13752449810769254 = (113.59270122120297 - 97.97092199705996) / 113.59270122120297 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: ShortSingleLine)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 103.37666760800555 < 108.92809833907287.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/31/2023 8:05:26 PM, 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 17.635983072865344 (T) = (0 -101.40217560606041) / Math.Sqrt((19.94217221594553 / (299)) + (7.920464556108737 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12510375098090312 = (115.90194348158309 - 101.40217560606041) / 115.90194348158309 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: ShortSingleLine)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 92.56779480550182 < 100.82459322762593.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 38.86232690929199 (T) = (0 -87.4709918245129) / Math.Sqrt((20.593039664160422 / (299)) + (3.0967825283996526 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19756331304855507 = (109.00672071316416 - 87.4709918245129) / 109.00672071316416 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FormatterDouble - Duration of single invocation 184.25 ns 169.44 ns 0.92 0.05 False
FormatterUInt32 - Duration of single invocation 8.82 ns 3.11 ns 0.35 0.06 False
FormatterDouble - Duration of single invocation 184.85 ns 169.36 ns 0.92 0.06 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: -1.7976931348623157E+308)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 169.44480653952965 < 175.24487391745535.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/17/2023 10:06:19 AM, 4/6/2023 1:55:21 PM, 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 25.80791071238084 (T) = (0 -169.71009409074375) / Math.Sqrt((27.22384871122557 / (299)) + (8.877393245800562 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11799556373169076 = (192.41410486411348 - 169.71009409074375) / 192.41410486411348 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 0)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.1141439916715834 < 8.383691591760781.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 8:24:48 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 7.908444629587516 (T) = (0 -3.0644963132138043) / Math.Sqrt((16.145241881347523 / (299)) + (0.0017127153170216885 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.37521192489773464 = (4.9048572393322445 - 3.0644963132138043) / 4.9048572393322445 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 1.7976931348623157E+308)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 169.3630408270688 < 175.37393234996503.
IsChangePoint: Marked as a change because one of 4/4/2023 8:16:00 PM, 4/24/2023 6:11:43 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 23.849294779837294 (T) = (0 -167.5173458732218) / Math.Sqrt((43.0961324004557 / (299)) + (12.046435913967812 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1284048234730665 = (192.19627458326727 - 167.5173458732218) / 192.19627458326727 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ArrayList>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeObjectProperty - Duration of single invocation 8.69 μs 5.64 μs 0.65 0.28 False
SerializeToStream - Duration of single invocation 7.41 μs 5.75 μs 0.78 0.37 False
SerializeToString - Duration of single invocation 7.77 μs 5.61 μs 0.72 0.35 False
SerializeToUtf8Bytes - Duration of single invocation 8.37 μs 5.66 μs 0.68 0.36 False
SerializeToStream - Duration of single invocation 7.17 μs 5.89 μs 0.82 0.17 False
SerializeToWriter - Duration of single invocation 7.09 μs 5.43 μs 0.77 0.37 False
SerializeToString - Duration of single invocation 7.62 μs 5.56 μs 0.73 0.30 False
SerializeObjectProperty - Duration of single invocation 7.72 μs 6.80 μs 0.88 0.35 False
SerializeToWriter - Duration of single invocation 8.10 μs 5.48 μs 0.68 0.39 False
SerializeToUtf8Bytes - Duration of single invocation 8.19 μs 6.37 μs 0.78 0.27 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ArrayList&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeObjectProperty(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.637895625765334 < 7.4918872423015.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 27.78662828091779 (T) = (0 -5917.9491463741415) / Math.Sqrt((264180.1258266744 / (299)) + (70058.00941103582 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2710670844516819 = (8118.647162369585 - 5917.9491463741415) / 8118.647162369585 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.750857492932645 < 7.037242822221244.
IsChangePoint: Marked as a change because one of 3/19/2023 2:30:51 PM, 4/26/2023 11:48:46 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 28.57493539981691 (T) = (0 -5561.280950060287) / Math.Sqrt((171792.1267665447 / (299)) + (79405.70714187472 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2957922445601089 = (7897.216279003306 - 5561.280950060287) / 7897.216279003306 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.6098963401073085 < 7.3398071690833175.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 23.03155534933758 (T) = (0 -6018.168750880626) / Math.Sqrt((215084.22839984787 / (299)) + (111576.97010185104 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26959653803849615 = (8239.512905263058 - 6018.168750880626) / 8239.512905263058 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.6556941269133185 < 7.721221660423331.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 23.418775718776164 (T) = (0 -5787.188615974489) / Math.Sqrt((259265.74269292384 / (299)) + (111518.14704357057 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.28227175359921053 = (8063.203092529317 - 5787.188615974489) / 8063.203092529317 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.892575157232702 < 6.985698879727797.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 4/12/2023 9:47:42 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 28.62472817824567 (T) = (0 -5602.151916030795) / Math.Sqrt((185934.92658265098 / (299)) + (79975.90873900625 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2960408780959236 = (7958.064242250364 - 5602.151916030795) / 7958.064242250364 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToWriter(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.425108104983234 < 6.738427595902461.
IsChangePoint: Marked as a change because one of 4/19/2023 4:18:46 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 28.543296439446284 (T) = (0 -5341.430271560007) / Math.Sqrt((453050.9362855779 / (299)) + (68529.56333177512 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3056650233976497 = (7692.87224690551 - 5341.430271560007) / 7692.87224690551 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.562760231316726 < 7.995791135594873.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 22.15838188690616 (T) = (0 -5940.670836471826) / Math.Sqrt((166717.27754048887 / (299)) + (141646.636121891 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2852968336738225 = (8312.081317631399 - 5940.670836471826) / 8312.081317631399 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeObjectProperty(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.804394725012402 < 7.545012527284025.
IsChangePoint: Marked as a change because one of 3/6/2023 4:48:38 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 20.73416624774281 (T) = (0 -5982.593114007788) / Math.Sqrt((184028.19067309937 / (299)) + (174107.4040334377 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2908783707215402 = (8436.624786209317 - 5982.593114007788) / 8436.624786209317 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToWriter(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.481332013147291 < 7.878603246868053.
IsChangePoint: Marked as a change because one of 4/21/2023 8:40:59 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 35.28038826826791 (T) = (0 -5218.437449836337) / Math.Sqrt((316152.350583595 / (299)) + (57362.61814296488 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.33333924984246033 = (7827.725643969829 - 5218.437449836337) / 7827.725643969829 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.3734963175122745 < 7.710728707423064.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 15.872850197913934 (T) = (0 -5983.456534029405) / Math.Sqrt((241437.34586245057 / (299)) + (225019.4559990421 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2631132555819031 = (8119.9133779430485 - 5983.456534029405) / 8119.9133779430485 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
JsonNet_ - Duration of single invocation 691.81 μs 483.09 μs 0.70 0.03 True
SystemTextJson_SourceGen_ - Duration of single invocation 308.99 μs 266.10 μs 0.86 0.07 False
SystemTextJson_Reflection_ - Duration of single invocation 361.36 μs 293.27 μs 0.81 0.06 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>.JsonNet_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 483.09475817724353 < 659.0896864525139.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 92.75672126524515 (T) = (0 -484023.1666995437) / Math.Sqrt((257291305.85556984 / (299)) + (50915861.73977405 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.29520548019102283 = (686757.8465716932 - 484023.1666995437) / 686757.8465716932 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>.SystemTextJson_SourceGen_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 266.1030015197569 < 296.33645446216275.
IsChangePoint: Marked as a change because one of 5/2/2023 10:35:24 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 29.54626492968187 (T) = (0 -261989.84884102212) / Math.Sqrt((90186620.44161433 / (299)) + (52469688.66982561 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1902063882361599 = (323526.69252400345 - 261989.84884102212) / 323526.69252400345 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>.SystemTextJson_Reflection_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 293.26565420560746 < 355.7041987126936.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 85.81365969638449 (T) = (0 -291196.3312634698) / Math.Sqrt((129035066.34363817 / (299)) + (9736696.52756876 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24253397128160045 = (384434.83961407706 - 291196.3312634698) / 384434.83961407706 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.StringReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLineAsync - Duration of single invocation 21.78 μs 18.74 μs 0.86 0.01 False
ReadLineAsync - Duration of single invocation 6.69 μs 5.60 μs 0.84 0.01 False
ReadLine - Duration of single invocation 15.23 μs 11.97 μs 0.79 0.02 False
ReadLine - Duration of single invocation 53.51 μs 43.43 μs 0.81 0.01 False
ReadLine - Duration of single invocation 83.38 μs 66.69 μs 0.80 0.01 True
ReadLineAsync - Duration of single invocation 77.16 μs 67.96 μs 0.88 0.01 False
ReadLine - Duration of single invocation 81.29 μs 69.32 μs 0.85 0.05 False
ReadLineAsync - Duration of single invocation 122.26 μs 102.80 μs 0.84 0.01 True
ReadLineAsync - Duration of single invocation 135.36 μs 103.96 μs 0.77 0.00 True
ReadLine - Duration of single invocation 4.68 μs 4.21 μs 0.90 0.02 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StringReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 9, 32])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.741369404476416 < 20.679976124364053.
IsChangePoint: Marked as a change because one of 3/21/2023 10:36:00 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 66.31104336447434 (T) = (0 -18821.415094955548) / Math.Sqrt((295683.99248311383 / (299)) + (20698.936964469653 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15181941851590924 = (22190.339540694353 - 18821.415094955548) / 22190.339540694353 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 33, 128])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.601888557726783 < 6.362812543780468.
IsChangePoint: Marked as a change because one of 5/1/2023 3:42:23 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 107.28168046124985 (T) = (0 -5619.514728800606) / Math.Sqrt((31690.406307519595 / (299)) + (411.6856038942944 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1829984363981421 = (6878.217838441144 - 5619.514728800606) / 6878.217838441144 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 9, 32])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.97246263501633 < 14.483077849865316.
IsChangePoint: Marked as a change because one of 3/21/2023 10:36:00 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 32.726237930030095 (T) = (0 -12012.693926114296) / Math.Sqrt((615599.5912808286 / (299)) + (119114.1834591665 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22395784008760014 = (15479.43468363922 - 12012.693926114296) / 15479.43468363922 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 1, 8])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 43.432133865438175 < 50.71454160826019.
IsChangePoint: Marked as a change because one of 3/21/2023 10:36:00 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 61.47457287141519 (T) = (0 -43352.788849368335) / Math.Sqrt((7721429.325752798 / (299)) + (142323.35290623648 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21378575840498124 = (55141.18996549478 - 43352.788849368335) / 55141.18996549478 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 0])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 66.69009118541034 < 79.28399387743792.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 92.61421457541724 (T) = (0 -66754.98980966902) / Math.Sqrt((9761000.984417448 / (299)) + (2722.745246664046 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20094265505207556 = (83542.17658060514 - 66754.98980966902) / 83542.17658060514 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 1, 8])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 67.96448650966937 < 73.13256923392699.
IsChangePoint: Marked as a change because one of 3/21/2023 10:36:00 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 21.187617023811274 (T) = (0 -69319.74461987906) / Math.Sqrt((7094793.914445132 / (299)) + (1942165.993588296 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11282730471365766 = (78135.57043423833 - 69319.74461987906) / 78135.57043423833 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 1, 1])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 69.32254545454545 < 77.19325291491438.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 39.180453013089085 (T) = (0 -69225.96070447152) / Math.Sqrt((3174559.5875665075 / (299)) + (1606884.9528479131 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17174341993214193 = (83580.33291905743 - 69225.96070447152) / 83580.33291905743 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 1, 1])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 102.79687206296994 < 116.31689820498512.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 82.25669755811829 (T) = (0 -103116.63085374658) / Math.Sqrt((3029673.671608086 / (299)) + (688421.0401523658 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16691833235603978 = (123777.3376353016 - 103116.63085374658) / 123777.3376353016 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 0, 0])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 103.9620669893021 < 128.6885516167.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 152.90970028867955 (T) = (0 -104530.2056645245) / Math.Sqrt((3878581.584685838 / (299)) + (457956.03173397895 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2430859497089156 = (138100.495854616 - 104530.2056645245) / 138100.495854616 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 33, 128])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.214570684523809 < 4.459198708424316.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 21.409802886141016 (T) = (0 -4124.493528304642) / Math.Sqrt((7357.10809925262 / (299)) + (12523.023217913164 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14026976640351208 = (4797.427573357228 - 4124.493528304642) / 4797.427573357228 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Ctor - Duration of single invocation 4.21 μs 3.48 μs 0.83 0.00 True
Ctor - Duration of single invocation 32.36 μs 27.87 μs 0.86 0.02 False
Ctor - Duration of single invocation 8.91 μs 7.56 μs 0.85 0.01 True
Ctor - Duration of single invocation 29.17 μs 24.47 μs 0.84 0.02 False
Ctor - Duration of single invocation 255.44 μs 228.10 μs 0.89 0.02 False
Count - Duration of single invocation 282.62 μs 261.76 μs 0.93 0.01 False
Count - Duration of single invocation 2.56 ms 2.35 ms 0.92 0.04 False
Ctor - Duration of single invocation 2.52 μs 2.06 μs 0.82 0.00 True
Ctor - Duration of single invocation 21.14 μs 18.30 μs 0.87 0.01 False
Count - Duration of single invocation 24.96 ms 21.41 ms 0.86 0.10 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Ctor(Pattern: "[\w]+://[^/\\s?#]+[^\\s?#]+(?:\?[^\\s#])?(?:#[^\\s])?", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.4802989594946125 < 3.9923811762067962.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 62.45104970803111 (T) = (0 -3457.0890810364353) / Math.Sqrt((2192.450482485704 / (299)) + (1575.5988365397354 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16998858211670617 = (4165.110270233089 - 3457.0890810364353) / 4165.110270233089 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Ctor(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 27.866339891074805 < 30.496132174299817.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 34.30369135927067 (T) = (0 -27839.65081981635) / Math.Sqrt((1407811.5132786003 / (299)) + (40048.51164166371 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09808142636267883 = (30867.144367082532 - 27839.65081981635) / 30867.144367082532 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Ctor(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.564898697318008 < 8.564583166415671.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 57.57615514504039 (T) = (0 -7636.7498162965785) / Math.Sqrt((10475.502388553583 / (299)) + (6502.159869876829 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14851480961100277 = (8968.740622262336 - 7636.7498162965785) / 8968.740622262336 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Ctor(Pattern: "[\w]+://[^/\\s?#]+[^\\s?#]+(?:\?[^\\s#])?(?:#[^\\s])?", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 24.466387353209598 < 27.675980790706845.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 30.771394403930614 (T) = (0 -24442.080548148177) / Math.Sqrt((2199149.7699052026 / (299)) + (47150.78236089155 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11655460126553864 = (27666.769879792846 - 24442.080548148177) / 27666.769879792846 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Ctor(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 228.09673758865247 < 242.89341716648522.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/20/2023 7:49:01 PM, 5/21/2023 3:58:09 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 18.04988585020083 (T) = (0 -227641.20814357512) / Math.Sqrt((29088106.90692469 / (299)) + (6344358.560549584 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10548934807773652 = (254486.8612289684 - 227641.20814357512) / 254486.8612289684 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w\.+-]+@[\w\.-]+\.[\w\.-]+", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 261.756962785114 < 265.528988694471.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 43.81455515703855 (T) = (0 -261716.9338275254) / Math.Sqrt((6864345.137950826 / (299)) + (2672102.4610477197 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07409674268987215 = (282661.20867513516 - 261716.9338275254) / 282661.20867513516 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w]+://[^/\\s?#]+[^\\s?#]+(?:\?[^\\s#])?(?:#[^\\s])?", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.353485786163522 < 2.4400662230017005.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 19.809726316109625 (T) = (0 -2154665.5626574624) / Math.Sqrt((1723251607.0022128 / (299)) + (5612663180.294925 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16128854873767734 = (2569018.891317785 - 2154665.5626574624) / 2569018.891317785 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Ctor(Pattern: "[\w\.+-]+@[\w\.-]+\.[\w\.-]+", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.0623385735866235 < 2.4106148734797936.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 70.49469151160498 (T) = (0 -2054.914440087595) / Math.Sqrt((857.2152424142101 / (299)) + (511.79483240195515 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18230351050156415 = (2513.0527848395827 - 2054.914440087595) / 2513.0527848395827 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Ctor(Pattern: "[\w\.+-]+@[\w\.-]+\.[\w\.-]+", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.30015134871232 < 19.791084207742237.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 32.177278244166835 (T) = (0 -18062.682740295728) / Math.Sqrt((368930.5542802435 / (299)) + (42927.91353991967 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10712605157615707 = (20229.823898637773 - 18062.682740295728) / 20229.823898637773 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.41044666666667 < 23.75978420833333.
IsChangePoint: Marked as a change because one of 3/28/2023 3:45:48 AM, 4/7/2023 7:53:47 AM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 24.75747167690896 (T) = (0 -22406206.762820516) / Math.Sqrt((1222742161903.0486 / (299)) + (132087850587.09917 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11653002195125475 = (25361593.850995872 - 22406206.762820516) / 25361593.850995872 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.BinaryWriterExtendedTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteAsciiCharArray - Duration of single invocation 40.44 ns 32.13 ns 0.79 0.08 False
WriteAsciiString - Duration of single invocation 14.91 ns 12.01 ns 0.81 0.02 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 32)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 32.12756292395806 < 37.14213484092221.
IsChangePoint: Marked as a change because one of 3/17/2023 9:25:32 PM, 3/23/2023 6:11:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 47.208660291207124 (T) = (0 -31.848183616466773) / Math.Sqrt((3.6557806707959197 / (299)) + (0.30353226279500495 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21849567412718024 = (40.75240860746006 - 31.848183616466773) / 40.75240860746006 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 32)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.008735886262121 < 14.167086745420292.
IsChangePoint: Marked as a change because one of 5/12/2023 4:02:48 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 93.30417149995873 (T) = (0 -11.998931412906048) / Math.Sqrt((1.0639758851724688 / (299)) + (0.0003810839573332313 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.31776411835548607 = (17.587658075067672 - 11.998931412906048) / 17.587658075067672 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromStream - Duration of single invocation 1.76 μs 1.15 μs 0.65 0.03 True
DeserializeFromString - Duration of single invocation 1.44 μs 945.64 ns 0.66 0.06 True
DeserializeFromUtf8Bytes - Duration of single invocation 1.26 μs 869.26 ns 0.69 0.06 True
DeserializeFromUtf8Bytes - Duration of single invocation 1.34 μs 932.93 ns 0.70 0.09 True
DeserializeFromStream - Duration of single invocation 1.81 μs 1.12 μs 0.62 0.05 True
DeserializeFromReader - Duration of single invocation 1.96 μs 1.26 μs 0.64 0.05 True
DeserializeFromReader - Duration of single invocation 1.98 μs 1.20 μs 0.60 0.05 True
DeserializeFromString - Duration of single invocation 1.45 μs 930.39 ns 0.64 0.04 True

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;LargeStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1499894837753444 < 1.6765707263133727.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 103.06123084544599 (T) = (0 -1142.7769949734686) / Math.Sqrt((815.505161241541 / (299)) + (418.61210203407273 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.34768208817488677 = (1751.8712490603014 - 1142.7769949734686) / 1751.8712490603014 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 945.6416596267029 < 1.3949066899210463.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 72.04736272928285 (T) = (0 -907.6790798964188) / Math.Sqrt((905.504307928482 / (299)) + (696.5800368698382 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3739142399435942 = (1449.767967593838 - 907.6790798964188) / 1449.767967593838 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 869.26053075314 < 1.1935756152049068.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 53.120601785039355 (T) = (0 -859.0566390742086) / Math.Sqrt((1176.817264407341 / (299)) + (651.3623188261392 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3125119224131807 = (1249.5585990227196 - 859.0566390742086) / 1249.5585990227196 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 932.9316814405619 < 1.2530210697594613.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 41.2649838963997 (T) = (0 -877.0624567179724) / Math.Sqrt((1169.4483502943367 / (299)) + (1220.3255875937639 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.31752017099053076 = (1285.1111775580453 - 877.0624567179724) / 1285.1111775580453 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1165607707409364 < 1.7104115835075508.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 52.641772094711605 (T) = (0 -1141.4604450929673) / Math.Sqrt((923.3692601537131 / (299)) + (1863.9618059651448 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.35820948159750926 = (1778.5561057122302 - 1141.4604450929673) / 1778.5561057122302 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromReader(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2563401000294205 < 1.8679403217959116.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 119.11349971160513 (T) = (0 -1239.0601893600508) / Math.Sqrt((2210.6167239980473 / (299)) + (308.62789355876833 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3491263101031693 = (1903.6876257149877 - 1239.0601893600508) / 1903.6876257149877 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromReader(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1957678694751381 < 1.8824896994586249.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 61.62362355749368 (T) = (0 -1261.2001298955292) / Math.Sqrt((2587.595283908698 / (299)) + (1325.5118087863227 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3394516958204055 = (1909.3230910068696 - 1261.2001298955292) / 1909.3230910068696 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 930.3912454102981 < 1.3602314428125721.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 83.52218764647169 (T) = (0 -937.1996308677207) / Math.Sqrt((1429.3400604121143 / (299)) + (360.39919328864124 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3369063141844157 = (1413.3743857249895 - 937.1996308677207) / 1413.3743857249895 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 3.07 ns 1.95 ns 0.64 0.07 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.ToString(value: 0)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.950017065802658 < 2.9103479233594465.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 5/1/2023 3:42:23 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 48.894306345736624 (T) = (0 -1.9530971592804762) / Math.Sqrt((0.06533287597072925 / (299)) + (0.0018720493113909256 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3227883721100242 = (2.8840277969913846 - 1.9530971592804762) / 2.8840277969913846 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateObject

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 24.29 μs 11.38 μs 0.47 0.26 True
Parse - Duration of single invocation 15.51 μs 8.46 μs 0.55 0.17 True
Parse - Duration of single invocation 686.65 μs 481.01 μs 0.70 0.18 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateObject*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_EnumerateObject.Parse(TestCase: NumericProperties)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.384230959317449 < 23.073716270377922.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 134.0478379146882 (T) = (0 -11129.414841139002) / Math.Sqrt((2010851.3337250932 / (299)) + (28383.79051665372 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5320154864099395 = (23781.587890081362 - 11129.414841139002) / 23781.587890081362 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_EnumerateObject.Parse(TestCase: StringProperties)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.455151196928636 < 14.995792011058544.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 155.49985741565567 (T) = (0 -8605.653667676603) / Math.Sqrt((406831.72960293945 / (299)) + (10878.62208260774 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.45859465763904217 = (15895.02909252633 - 8605.653667676603) / 15895.02909252633 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_EnumerateObject.Parse(TestCase: ObjectProperties)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 481.01021464646453 < 656.8854769021739.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 36.782536592413784 (T) = (0 -469018.54819047364) / Math.Sqrt((435692441.09884006 / (299)) + (440493682.69185346 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.31797510199122775 = (687685.375650965 - 469018.54819047364) / 687685.375650965 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 66.51 ns 47.89 ns 0.72 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int16>.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 47.886987681560605 < 61.87687691644979.
IsChangePoint: Marked as a change because one of 5/4/2023 10:11:14 AM, 5/9/2023 1:22:51 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 86.91230932138112 (T) = (0 -45.81848051197054) / Math.Sqrt((7.914166824231077 / (299)) + (0.4666199127531446 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3214355481685008 = (67.522665515859 - 45.81848051197054) / 67.522665515859 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractSerializer_ - Duration of single invocation 1.58 ms 1.32 ms 0.83 0.06 False
XmlSerializer_ - Duration of single invocation 556.68 μs 457.52 μs 0.82 0.16 False
DataContractSerializer_BinaryXml_ - Duration of single invocation 481.49 μs 407.76 μs 0.85 0.13 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;CollectionsOfPrimitives&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>.DataContractSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3213037943262413 < 1.487814839542009.
IsChangePoint: Marked as a change because one of 3/27/2023 11:28:03 AM, 3/31/2023 8:05:26 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 20.380722344531964 (T) = (0 -1314080.2605706172) / Math.Sqrt((1614762979.8342698 / (299)) + (2217227413.0684295 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17062780681735445 = (1584427.68080751 - 1314080.2605706172) / 1584427.68080751 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>.XmlSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 457.522271062271 < 535.0947153191138.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 29.325136343759578 (T) = (0 -454745.2116163) / Math.Sqrt((445171541.2000883 / (299)) + (177842456.88923833 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20074197873514174 = (568959.209063235 - 454745.2116163) / 568959.209063235 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>.DataContractSerializer_BinaryXml_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 407.76264775413716 < 457.00949131990853.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 35.01189165257006 (T) = (0 -399381.3650061352) / Math.Sqrt((483579555.5712133 / (299)) + (99792211.87664449 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2108909286509009 = (506116.8088250886 - 399381.3650061352) / 506116.8088250886 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Cache

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsMatch - Duration of single invocation 10.78 ms 8.96 ms 0.83 0.06 False
IsMatch - Duration of single invocation 54.22 ms 36.51 ms 0.67 0.31 False
IsMatch - Duration of single invocation 45.69 ms 34.20 ms 0.75 0.28 False
IsMatch_Multithreading - Duration of single invocation 17.26 ms 12.00 ms 0.70 0.34 False
IsMatch - Duration of single invocation 111.26 ms 94.54 ms 0.85 0.07 False
IsMatch_Multithreading - Duration of single invocation 36.61 ms 24.49 ms 0.67 0.70 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Cache*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total: 40000, unique: 1600, cacheSize: 3200)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.955375510204082 < 10.102137949275361.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 6.303319783114112 (T) = (0 -9294517.92805485) / Math.Sqrt((73618494479.19574 / (299)) + (612128298971.3793 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12857448535485017 = (10665877.658906555 - 9294517.92805485) / 10665877.658906555 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total: 400000, unique: 7, cacheSize: 15)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 36.50992857142858 < 51.531952296875.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 13.142549590621936 (T) = (0 -41243106.8568611) / Math.Sqrt((8977345798985.84 / (299)) + (11256851779933.988 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23172935106296044 = (53683043.747570015 - 41243106.8568611) / 53683043.747570015 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total: 400000, unique: 1, cacheSize: 15)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 34.203698901098896 < 42.07401879166666.
IsChangePoint: Marked as a change because one of 3/31/2023 8:05:26 PM, 4/9/2023 11:04:55 PM, 4/24/2023 11:08:13 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 39.27627840877765 (T) = (0 -30421412.192695122) / Math.Sqrt((8042481219883.086 / (299)) + (1299418343353.7087 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.31499089274035624 = (44410230.27328056 - 30421412.192695122) / 44410230.27328056 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch_Multithreading(total: 40000, unique: 7, cacheSize: 0)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.001193859649122 < 15.268360735653237.
IsChangePoint: Marked as a change because one of 5/19/2023 3:32:16 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 7.366647040889252 (T) = (0 -12667948.784983292) / Math.Sqrt((785980141526.4965 / (299)) + (2123246803813.6023 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.19770206543446814 = (15789581.699276544 - 12667948.784983292) / 15789581.699276544 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total: 40000, unique: 1600, cacheSize: 15)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 94.5354 < 105.70269177884616.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 49.243318769053054 (T) = (0 -94249671.00591716) / Math.Sqrt((25620956958094.543 / (299)) + (702134599588.2719 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16337802984554203 = (112655027.44150586 - 94249671.00591716) / 112655027.44150586 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch_Multithreading(total: 400000, unique: 1, cacheSize: 15)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 24.48542875 < 32.81327274080882.
IsChangePoint: Marked as a change because one of 3/15/2023 4:14:49 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 17.32826602052581 (T) = (0 -24149083.608299594) / Math.Sqrt((23821123622255.367 / (299)) + (1455871614479.4834 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23904452370941803 = (31735212.322827037 - 24149083.608299594) / 31735212.322827037 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dequeue_And_Enqueue - Duration of single invocation 289.10 μs 265.05 μs 0.92 0.00 True
HeapSort - Duration of single invocation 3.89 μs 3.34 μs 0.86 0.03 False
HeapSort - Duration of single invocation 218.68 ns 182.94 ns 0.84 0.05 True
K_Max_Elements - Duration of single invocation 911.21 ns 610.71 ns 0.67 0.08 True
Enumerate - Duration of single invocation 3.34 μs 2.46 μs 0.74 0.01 True
K_Max_Elements - Duration of single invocation 6.65 μs 3.72 μs 0.56 0.00 True
Dequeue_And_Enqueue - Duration of single invocation 636.92 ns 459.75 ns 0.72 0.39 False
HeapSort - Duration of single invocation 113.84 μs 102.49 μs 0.90 0.01 True

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Guid, Guid&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Dequeue_And_Enqueue(Size: 1000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 265.0542025862069 < 272.5419554924242.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 18.951052503470777 (T) = (0 -267082.55032673123) / Math.Sqrt((4105851.7310026386 / (299)) + (15040887.197568014 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07130013127798884 = (287587.58272924594 - 267082.55032673123) / 287587.58272924594 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.HeapSort(Size: 100)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.341954991087345 < 3.703746560583457.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/9/2023 11:04:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 37.84232365488323 (T) = (0 -3294.3803430661364) / Math.Sqrt((14156.634421885552 / (299)) + (3711.0647233475793 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1732517274855772 = (3984.7441507761546 - 3294.3803430661364) / 3984.7441507761546 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.HeapSort(Size: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 182.93638522331787 < 206.3253732137893.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 27.18438757685879 (T) = (0 -166.5627059226772) / Math.Sqrt((21.73895075141947 / (299)) + (45.35677896266705 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23548189770548372 = (217.86626820578806 - 166.5627059226772) / 217.86626820578806 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.K_Max_Elements(Size: 100)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 610.7078436415918 < 874.8234306576562.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 89.7193539200966 (T) = (0 -600.9832299401374) / Math.Sqrt((432.3371029626327 / (299)) + (164.99802411436917 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.35952096172014353 = (938.3339563371294 - 600.9832299401374) / 938.3339563371294 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Enumerate(Size: 1000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.4612756864884524 < 3.2199618839922612.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 336.2873017197843 (T) = (0 -2464.9417279130994) / Math.Sqrt((1585.125315338513 / (299)) + (26.000206147545146 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2693501951984916 = (3373.6294894142025 - 2464.9417279130994) / 3373.6294894142025 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.K_Max_Elements(Size: 1000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.71545910311069 < 6.315586449013191.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 321.52327935050397 (T) = (0 -3676.057436488843) / Math.Sqrt((4106.187325083293 / (299)) + (912.8240855802036 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4448716219659163 = (6621.99516714878 - 3676.057436488843) / 6621.99516714878 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Dequeue_And_Enqueue(Size: 10)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 459.74807614992886 < 604.5317575834468.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 13.598956467794986 (T) = (0 -497.62527273881585) / Math.Sqrt((572.3550015576591 / (299)) + (901.595607126821 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18745501358098618 = (612.4279652895428 - 497.62527273881585) / 612.4279652895428 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.HeapSort(Size: 1000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 102.4949623655914 < 108.74394163602942.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 59.887028171893995 (T) = (0 -101561.06445778593) / Math.Sqrt((1180447.0310005278 / (299)) + (592453.0439933209 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11599900551725507 = (114887.95271911692 - 101561.06445778593) / 114887.95271911692 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.BinaryWriterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteUInt16 - Duration of single invocation 1.95 ns 0.85 ns 0.44 0.07 False
WriteUInt32 - Duration of single invocation 1.95 ns 0.85 ns 0.44 0.04 False
WriteUInt64 - Duration of single invocation 1.95 ns 0.86 ns 0.44 0.06 False
WriteDouble - Duration of single invocation 2.23 ns 1.11 ns 0.50 0.05 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryWriterTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.BinaryWriterTests.WriteUInt16

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.8531242052997501 < 1.8520079658329838.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 112.96517839646951 (T) = (0 -0.8603916753399385) / Math.Sqrt((0.0161738574955142 / (299)) + (0.0006456636516501492 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5721749016395463 = (2.0110827500237876 - 0.8603916753399385) / 2.0110827500237876 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.BinaryWriterTests.WriteUInt32

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.852900720248604 < 1.8521617120384266.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 165.65967671402046 (T) = (0 -0.8527682896781089) / Math.Sqrt((0.01451523053883488 / (299)) + (1.7196054586445134E-07 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5751363840211628 = (2.007157726870606 - 0.8527682896781089) / 2.007157726870606 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.BinaryWriterTests.WriteUInt64

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.8613890397017773 < 1.8515586402937176.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 163.2242337939021 (T) = (0 -0.8615723130413413) / Math.Sqrt((0.01474470361264933 / (299)) + (4.636750730656436E-07 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5709737825978572 = (2.00820434298484 - 0.8615723130413413) / 2.00820434298484 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.BinaryWriterTests.WriteDouble

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1144171253628699 < 2.116577938355055.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 4/12/2023 4:44:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 120.10026170006003 (T) = (0 -1.114006316042697) / Math.Sqrt((0.021490313361109258 / (299)) + (8.247683879976994E-07 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.47764166439060557 = (2.132647724943593 - 1.114006316042697) / 2.132647724943593 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 18.30 μs 11.37 μs 0.62 0.08 False
SortedDictionary - Duration of single invocation 33.67 μs 16.84 μs 0.50 0.08 True
ImmutableDictionary - Duration of single invocation 11.98 μs 6.82 μs 0.57 0.25 True
IDictionary - Duration of single invocation 3.60 μs 2.75 μs 0.76 0.05 False
ImmutableSortedDictionary - Duration of single invocation 25.57 μs 11.16 μs 0.44 0.17 True

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyTrue<Int32, Int32>.SortedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.373791195073236 < 17.32809261630676.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 47.51503526195144 (T) = (0 -11280.091860701446) / Math.Sqrt((2245196.7582433466 / (299)) + (292971.9767166273 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4220125027667255 = (19516.152018334793 - 11280.091860701446) / 19516.152018334793 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsKeyTrue<Int32, Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.84252591928436 < 32.6593417947665.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 189.04556374881375 (T) = (0 -16671.031687783663) / Math.Sqrt((1885346.7318469698 / (299)) + (44475.855667743024 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5279400078408142 = (35315.49371835336 - 16671.031687783663) / 35315.49371835336 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsKeyTrue<Int32, Int32>.ImmutableDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.824854095925297 < 10.948662498319257.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 36.65884027465409 (T) = (0 -7043.198200729745) / Math.Sqrt((166609.42047778037 / (299)) + (162256.07118434916 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3727742008700619 = (11229.127071143726 - 7043.198200729745) / 11229.127071143726 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsKeyTrue<Int32, Int32>.IDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.750544371167418 < 3.409125296709828.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 38.81340024652091 (T) = (0 -2667.0013603330863) / Math.Sqrt((60454.83245361958 / (299)) + (9308.42722392607 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3060345071044937 = (3843.1325298398797 - 2667.0013603330863) / 3843.1325298398797 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsKeyTrue<Int32, Int32>.ImmutableSortedDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.1599331137067 < 23.89848760967577.
IsChangePoint: Marked as a change because one of 5/9/2023 7:00:05 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 141.87602700139956 (T) = (0 -11139.355409693286) / Math.Sqrt((1584829.216370156 / (299)) + (8437.080912579955 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4955601243093786 = (22082.622620669223 - 11139.355409693286) / 22082.622620669223 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromReader - Duration of single invocation 71.27 μs 63.72 μs 0.89 0.16 False
DeserializeFromUtf8Bytes - Duration of single invocation 68.92 μs 61.63 μs 0.89 0.14 False
DeserializeFromReader - Duration of single invocation 72.21 μs 66.14 μs 0.92 0.21 False
DeserializeFromStream - Duration of single invocation 68.19 μs 61.50 μs 0.90 0.18 False
DeserializeFromStream - Duration of single invocation 69.67 μs 60.92 μs 0.87 0.10 False
DeserializeFromString - Duration of single invocation 68.48 μs 61.80 μs 0.90 0.15 False
DeserializeFromString - Duration of single invocation 68.48 μs 61.08 μs 0.89 0.19 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromReader(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 63.71824968730457 < 74.30077436133875.
IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 7.7157422113987915 (T) = (0 -66438.35719100085) / Math.Sqrt((5882459.095344905 / (299)) + (19571238.215466138 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1254322165076996 = (75967.0759031404 - 66438.35719100085) / 75967.0759031404 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 61.62616662851038 < 67.93444911186317.
IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 20.061026770478634 (T) = (0 -61759.10498731639) / Math.Sqrt((4572302.412699583 / (299)) + (2339992.875986932 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12552776219349843 = (70624.43187702673 - 61759.10498731639) / 70624.43187702673 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromReader(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 66.13718223836248 < 72.87947680277738.
IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 32.935906515062115 (T) = (0 -64647.01589131269) / Math.Sqrt((7760744.398663122 / (299)) + (1307979.8777592944 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15344144509208807 = (76364.49424144671 - 64647.01589131269) / 76364.49424144671 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 61.49964285714285 < 71.29190198050648.
IsChangePoint: Marked as a change because one of 5/13/2023 6:35:24 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.010971557747867 (T) = (0 -63241.33463257403) / Math.Sqrt((5417677.285375872 / (299)) + (8745899.569513554 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1363395115728805 = (73224.76306372175 - 63241.33463257403) / 73224.76306372175 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 60.92077039808019 < 69.04951455306534.
IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 50.755241670939895 (T) = (0 -61566.949474610534) / Math.Sqrt((4556908.555773025 / (299)) + (453051.7196299255 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15576650825805685 = (72926.44757266951 - 61566.949474610534) / 72926.44757266951 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 61.80334203839639 < 66.63571340101997.
IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 21.853928473076632 (T) = (0 -61539.18180992749) / Math.Sqrt((4862323.203052003 / (299)) + (2454463.7977314545 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1385360075631324 = (71435.5821603738 - 61539.18180992749) / 71435.5821603738 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 61.08382975260417 < 68.79269192035837.
IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 5/19/2023 10:55:27 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.71297640963141 (T) = (0 -61674.050821517034) / Math.Sqrt((6794855.081898676 / (299)) + (5837874.296056501 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.13832706643965711 = (71574.78019726844 - 61674.050821517034) / 71574.78019726844 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableHashSet - Duration of single invocation 10.42 μs 7.18 μs 0.69 0.07 False
SortedSet - Duration of single invocation 25.46 μs 12.73 μs 0.50 0.03 True
ImmutableSortedSet - Duration of single invocation 23.41 μs 14.21 μs 0.61 0.17 False
Queue - Duration of single invocation 19.93 μs 17.28 μs 0.87 0.02 True

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.ImmutableHashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.177149643995059 < 10.008303585902699.
IsChangePoint: Marked as a change because one of 4/19/2023 8:38:27 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 15.91911841841969 (T) = (0 -7856.135374832259) / Math.Sqrt((124228.98297124029 / (299)) + (226658.8528660154 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2130516328376502 = (9983.037900136484 - 7856.135374832259) / 9983.037900136484 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsFalse<Int32>.SortedSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.727579727235437 < 24.209049334961996.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 92.3349452867985 (T) = (0 -12234.571261050209) / Math.Sqrt((343160.1167614063 / (299)) + (266898.520518723 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5263347008933561 = (25829.57055145313 - 12234.571261050209) / 25829.57055145313 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsFalse<Int32>.ImmutableSortedSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.20595452209661 < 22.353910432017337.
IsChangePoint: Marked as a change because one of 4/24/2023 11:08:13 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 128.39694189063638 (T) = (0 -14164.328545702376) / Math.Sqrt((2282161.320021614 / (299)) + (3607.4339641190104 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.44635601959056503 = (25583.8210960543 - 14164.328545702376) / 25583.8210960543 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsFalse<Int32>.Queue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.27996573523162 < 18.932738788986576.
IsChangePoint: Marked as a change because one of 5/1/2023 3:42:23 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 165.53318244176944 (T) = (0 -17270.981298041843) / Math.Sqrt((101228.78150489269 / (299)) + (69.1186396801779 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1509111261845727 = (20340.60488913692 - 17270.981298041843) / 20340.60488913692 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LinkedList - Duration of single invocation 666.42 μs 632.75 μs 0.95 0.03 False
Span - Duration of single invocation 434.81 μs 365.22 μs 0.84 0.29 False
ImmutableHashSet - Duration of single invocation 33.95 μs 30.93 μs 0.91 0.02 False
ICollection - Duration of single invocation 452.44 μs 412.15 μs 0.91 0.12 False
Array - Duration of single invocation 344.41 μs 203.08 μs 0.59 0.06 True

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<String>.LinkedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 632.7483833333333 < 633.533031567174.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 11.105549787663007 (T) = (0 -601932.3304997236) / Math.Sqrt((351894923.7383325 / (299)) + (185978722.04211655 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06768349619085415 = (645630.886121206 - 601932.3304997236) / 645630.886121206 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<String>.Span(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 365.2238683554817 < 407.2556811052124.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 4.948775305068164 (T) = (0 -375730.7478901712) / Math.Sqrt((789263400.7534593 / (299)) + (308789629.4414852 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06337648857043628 = (401154.51225081383 - 375730.7478901712) / 401154.51225081383 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<String>.ImmutableHashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 30.926380157170918 < 32.34657028695478.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 21.888732883552546 (T) = (0 -30333.92839860419) / Math.Sqrt((380063.36201039463 / (299)) + (435190.76674066926 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11856174068436136 = (34414.127226739496 - 30333.92839860419) / 34414.127226739496 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<String>.ICollection(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 412.15442708333325 < 429.521484077381.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/20/2023 7:49:01 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 16.692933841877004 (T) = (0 -411923.4780876757) / Math.Sqrt((540010958.3956378 / (299)) + (2666933.2794678793 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.05730981666286749 = (436965.91453775676 - 411923.4780876757) / 436965.91453775676 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<String>.Array(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 203.07743506493506 < 327.7142308650362.
IsChangePoint: Marked as a change because one of 5/16/2023 12:30:38 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 193.34722811735057 (T) = (0 -205777.01796874683) / Math.Sqrt((134904072.510893 / (299)) + (1306411.6211884718 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4110318543674891 = (349385.6492828769 - 205777.01796874683) / 349385.6492828769 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 3.24 ns 1.95 ns 0.60 0.04 False
ToStringHex - Duration of single invocation 17.04 ns 15.23 ns 0.89 0.05 False
ToStringHex - Duration of single invocation 14.59 ns 13.00 ns 0.89 0.13 False
ToStringHex - Duration of single invocation 20.21 ns 18.79 ns 0.93 0.03 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.ToString(value: 4)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.9515674415963378 < 3.0761245629079665.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 36.841653323202344 (T) = (0 -1.9248471357499177) / Math.Sqrt((0.1809618847161259 / (299)) + (0.0017621285110644364 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3425105464895874 = (2.9275711199212933 - 1.9248471357499177) / 2.9275711199212933 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ToStringHex(value: 12345)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 15.226563521810288 < 16.216653673454775.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/31/2023 8:05:26 PM, 4/9/2023 11:04:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 11.308818478053345 (T) = (0 -15.701486545808992) / Math.Sqrt((0.4163321298221645 / (299)) + (0.1683602774484828 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0794084231912656 = (17.05586596853166 - 15.701486545808992) / 17.05586596853166 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ToStringHex(value: 4)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.998955417098369 < 14.388800273561673.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/25/2023 6:46:56 PM, 5/16/2023 5:40:18 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.154646573739646 (T) = (0 -13.379220123839003) / Math.Sqrt((0.8050589113034262 / (299)) + (0.24474428755687136 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11759537403202244 = (15.162227996212403 - 13.379220123839003) / 15.162227996212403 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ToStringHex(value: -2147483648)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.79047810777034 < 19.325423954487455.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/17/2023 6:40:18 PM, 4/19/2023 1:39:34 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 11.349175406331172 (T) = (0 -19.044688293333504) / Math.Sqrt((0.366630297263956 / (299)) + (0.14846387147275192 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06280650681502468 = (20.320977932328244 - 19.044688293333504) / 20.320977932328244 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToWriter - Duration of single invocation 55.74 ns 29.19 ns 0.52 0.12 False
SerializeObjectProperty - Duration of single invocation 295.08 ns 201.45 ns 0.68 0.11 False
SerializeToStream - Duration of single invocation 193.87 ns 138.28 ns 0.71 0.09 False
SerializeToStream - Duration of single invocation 200.08 ns 150.68 ns 0.75 0.10 False
SerializeToUtf8Bytes - Duration of single invocation 124.06 ns 75.63 ns 0.61 0.07 False
SerializeToWriter - Duration of single invocation 54.16 ns 29.84 ns 0.55 0.12 False
SerializeToString - Duration of single invocation 137.02 ns 99.42 ns 0.73 0.05 False
SerializeToUtf8Bytes - Duration of single invocation 121.93 ns 81.25 ns 0.67 0.06 False
SerializeObjectProperty - Duration of single invocation 296.25 ns 209.89 ns 0.71 0.07 False
SerializeToString - Duration of single invocation 140.64 ns 104.07 ns 0.74 0.08 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToWriter(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 29.190143154820188 < 52.8679712756076.
IsChangePoint: Marked as a change because one of 3/20/2023 4:51:16 PM, 4/17/2023 1:30:24 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 25.581470223664322 (T) = (0 -30.35042970842525) / Math.Sqrt((22.77007132724688 / (299)) + (7.9614993279427155 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.41156333213785556 = (51.57807350567007 - 30.35042970842525) / 51.57807350567007 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeObjectProperty(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 201.44835854389865 < 293.86004409152065.
IsChangePoint: Marked as a change because one of 4/18/2023 4:36:42 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 93.509272461955 (T) = (0 -203.50692884621637) / Math.Sqrt((160.901617668866 / (299)) + (4.740688269571916 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.303906033240816 = (292.3555418727257 - 203.50692884621637) / 292.3555418727257 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 138.28382077452278 < 179.49172005455242.
IsChangePoint: Marked as a change because one of 4/19/2023 1:39:34 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 47.11826257189715 (T) = (0 -144.95197241721456) / Math.Sqrt((58.997682429986206 / (299)) + (12.038006305107402 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2562409765739049 = (194.89104380811315 - 144.95197241721456) / 194.89104380811315 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 150.67866101702637 < 190.46176432961136.
IsChangePoint: Marked as a change because one of 4/19/2023 1:39:34 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 31.24062583784743 (T) = (0 -147.6746550358138) / Math.Sqrt((56.57874992031737 / (299)) + (26.475912554951808 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2399003159934349 = (194.2832738166726 - 147.6746550358138) / 194.2832738166726 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 75.62627677486473 < 122.60566744434405.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/19/2023 3:30:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 104.72665028027372 (T) = (0 -76.59463095779068) / Math.Sqrt((61.63608775343729 / (299)) + (0.24299047441753682 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3933210545553331 = (126.25233087930957 - 76.59463095779068) / 126.25233087930957 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToWriter(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 29.839240313265126 < 51.57309308828577.
IsChangePoint: Marked as a change because one of 3/20/2023 11:25:43 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 75.75527778116806 (T) = (0 -29.123263533569116) / Math.Sqrt((19.23973889680794 / (299)) + (0.2934749296488283 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.43403635843083965 = (51.45783473444252 - 29.123263533569116) / 51.45783473444252 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 99.42130113405825 < 137.95027744418778.
IsChangePoint: Marked as a change because one of 3/20/2023 4:51:16 PM, 4/17/2023 6:40:18 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 26.94278883891303 (T) = (0 -100.41620950802711) / Math.Sqrt((43.79630343404787 / (299)) + (33.202644438223736 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3060004445891719 = (144.69203722844398 - 100.41620950802711) / 144.69203722844398 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 81.253023838152 < 120.40529909692204.
IsChangePoint: Marked as a change because one of 3/23/2023 6:11:55 PM, 4/19/2023 8:38:27 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 67.10594128973244 (T) = (0 -80.88854422211635) / Math.Sqrt((37.41675086182533 / (299)) + (4.568868429290075 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3641613332417122 = (127.21551621657808 - 80.88854422211635) / 127.21551621657808 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeObjectProperty(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 209.89254117566736 < 291.2795463334378.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/10/2023 4:31:44 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 68.51537369693337 (T) = (0 -210.86483746966672) / Math.Sqrt((171.61560139725125 / (299)) + (14.134400501170257 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2951750276436535 = (299.1733348560433 - 210.86483746966672) / 299.1733348560433 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 104.06776487279285 < 140.44490267284522.
IsChangePoint: Marked as a change because one of 3/22/2023 6:15:16 AM, 4/19/2023 3:30:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 70.25488172967681 (T) = (0 -104.5359671984202) / Math.Sqrt((58.83658939613831 / (299)) + (1.9190387784510181 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2828471028728146 = (145.76524422780236 - 104.5359671984202) / 145.76524422780236 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchF.Secant

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 180.67 ms 131.93 ms 0.73 0.00 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.Secant*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchF.Secant.Test

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 131.92737666666667 < 171.59675516666664.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 3987.819260359799 (T) = (0 -131872411.33699633) / Math.Sqrt((30675907404.547607 / (299)) + (604800824.1359186 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26968509047739153 = (180569244.3321451 - 131872411.33699633) / 180569244.3321451 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SingleSerial - Duration of single invocation 24.56 ns 22.52 ns 0.92 0.08 False
SingleSerial - Duration of single invocation 598.25 ns 554.05 ns 0.93 0.03 False
MultipleSerial - Duration of single invocation 233.25 ns 208.13 ns 0.89 0.11 False
MultipleSerial - Duration of single invocation 518.79 ns 454.50 ns 0.88 0.25 False
MultipleSerial - Duration of single invocation 491.90 ns 402.82 ns 0.82 0.16 False

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: False, UseSharedPool: False)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 22.524604395604392 < 23.54825474959936.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 4/17/2023 7:48:00 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 29.381453773662738 (T) = (0 -22.550666162877874) / Math.Sqrt((3.025683737235812 / (299)) + (0.004209892905076987 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11750171090469222 = (25.553212330865442 - 22.550666162877874) / 25.553212330865442 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 554.0510892857142 < 571.0008784375.
IsChangePoint: Marked as a change because one of 3/16/2023 12:15:41 AM, 4/28/2023 10:42:00 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 19.033246966505864 (T) = (0 -555.6802102071007) / Math.Sqrt((287.8303789254337 / (299)) + (115.56400835500743 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09707468828045884 = (615.4221207387099 - 555.6802102071007) / 615.4221207387099 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.MultipleSerial(RentalSize: 4096, ManipulateArray: False, Async: False, UseSharedPool: False)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 208.12729411764707 < 221.86883854166666.
IsChangePoint: Marked as a change because one of 3/5/2023 11:36:44 PM, 3/17/2023 9:25:32 PM, 3/20/2023 8:31:51 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 28.70316385707802 (T) = (0 -207.1328060743775) / Math.Sqrt((168.08304559553255 / (299)) + (6.742213497971875 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12592164579869589 = (236.97281265321655 - 207.1328060743775) / 236.97281265321655 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.MultipleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 454.50044642857137 < 480.7238040972222.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 10.045103226038728 (T) = (0 -467.67625833849735) / Math.Sqrt((462.9048135239256 / (299)) + (295.7906987301701 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.09278995350530063 = (515.5104489258209 - 467.67625833849735) / 515.5104489258209 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.MultipleSerial(RentalSize: 4096, ManipulateArray: False, Async: False, UseSharedPool: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 402.82086111111107 < 428.67045937499995.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 10.826199658460224 (T) = (0 -416.2684887124739) / Math.Sqrt((320.58468046944364 / (299)) + (94.72208665827434 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06993282738864527 = (447.56819826649144 - 416.2684887124739) / 447.56819826649144 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Security.Cryptography.Tests.Perf_CryptoConfig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateFromName - Duration of single invocation 318.48 ns 281.28 ns 0.88 0.10 False
CreateFromName - Duration of single invocation 387.98 ns 358.38 ns 0.92 0.10 False
CreateFromName - Duration of single invocation 751.52 ns 674.09 ns 0.90 0.23 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Security.Cryptography.Tests.Perf_CryptoConfig*'

Payloads

Baseline
Compare

Histogram

System.Security.Cryptography.Tests.Perf_CryptoConfig.CreateFromName(name: "X509Chain")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 281.2754370608846 < 303.0841483238786.
IsChangePoint: Marked as a change because one of 3/8/2023 4:46:51 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 27.151348336627077 (T) = (0 -281.30654382690454) / Math.Sqrt((89.41104348178 / (299)) + (33.95658542150831 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1413944115953839 = (327.63185754428076 - 281.30654382690454) / 327.63185754428076 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Security.Cryptography.Tests.Perf_CryptoConfig.CreateFromName(name: "RSA")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 358.3834434331825 < 373.0601440916574.
IsChangePoint: Marked as a change because one of 4/24/2023 6:11:43 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 19.258408529025086 (T) = (0 -340.20592836881593) / Math.Sqrt((168.67752444343634 / (299)) + (116.04236135060007 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1484943740008431 = (399.5345632268938 - 340.20592836881593) / 399.5345632268938 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Security.Cryptography.Tests.Perf_CryptoConfig.CreateFromName(name: "SHA512")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 674.0860874841475 < 710.9177616852201.
IsChangePoint: Marked as a change because one of 3/26/2023 7:47:31 AM, 3/27/2023 8:33:24 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 20.78279493155153 (T) = (0 -676.3071138654725) / Math.Sqrt((1610.5660856421255 / (299)) + (202.27362728887258 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12329966831454428 = (771.4233580421632 - 676.3071138654725) / 771.4233580421632 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitArrayLeftShift - Duration of single invocation 4.46 ns 3.37 ns 0.75 0.05 False
BitArraySetAll - Duration of single invocation 3.90 ns 2.75 ns 0.70 0.09 False
BitArrayCopyToIntArray - Duration of single invocation 16.28 ns 12.30 ns 0.76 0.05 False
BitArrayCopyToIntArray - Duration of single invocation 9.05 ns 6.52 ns 0.72 0.06 False
BitArrayLengthValueCtor - Duration of single invocation 13.28 ns 11.87 ns 0.89 0.05 False
BitArrayLengthValueCtor - Duration of single invocation 14.71 ns 13.19 ns 0.90 0.06 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArrayLeftShift(Size: 4)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.3666547982549835 < 4.239794525813045.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 78.44448273610706 (T) = (0 -3.359944833572523) / Math.Sqrt((0.018030375156571264 / (299)) + (0.0008226308428123545 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20606040293882244 = (4.231990501556531 - 3.359944833572523) / 4.231990501556531 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 4)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.746078228739211 < 3.7044425437012056.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM, 5/9/2023 1:22:51 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 47.91927359894697 (T) = (0 -2.756459844397118) / Math.Sqrt((0.09487257199852031 / (299)) + (0.0003590873464231111 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24406404374486698 = (3.6464198079060495 - 2.756459844397118) / 3.6464198079060495 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayCopyToIntArray(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.303562522185725 < 15.463616850934834.
IsChangePoint: Marked as a change because one of 5/2/2023 3:18:41 PM, 5/5/2023 3:18:41 PM, 5/10/2023 6:05:58 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 46.80511025738034 (T) = (0 -12.31971616757396) / Math.Sqrt((1.2001247163810094 / (299)) + (0.0010625929008636608 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1955822462789075 = (15.31507243666013 - 12.31971616757396) / 15.31507243666013 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayCopyToIntArray(Size: 4)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.521680975966062 < 8.591367912271565.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/9/2023 11:04:55 PM, 4/17/2023 6:40:18 PM, 5/5/2023 3:18:41 PM, 5/10/2023 6:05:58 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 72.87484077800953 (T) = (0 -6.537396188619535) / Math.Sqrt((0.4114862324944232 / (299)) + (0.00034577966729880524 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2945399142467533 = (9.26685480956631 - 6.537396188619535) / 9.26685480956631 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayLengthValueCtor(Size: 4)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.869631329393677 < 14.56385769719604.
IsChangePoint: Marked as a change because one of 3/3/2023 11:43:56 AM, 3/31/2023 8:05:26 PM, 4/3/2023 9:14:01 PM, 5/11/2023 5:02:01 PM, 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 30.664504505572886 (T) = (0 -11.685204662450314) / Math.Sqrt((0.34824562590114727 / (299)) + (0.03691289854506786 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1424083916802149 = (13.625605182103238 - 11.685204662450314) / 13.625605182103238 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_BitArray.BitArrayLengthValueCtor(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.193237300036943 < 15.93026980082535.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/11/2023 5:02:01 PM, 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 16.15566407491692 (T) = (0 -12.88970824129047) / Math.Sqrt((0.3724955945893149 / (299)) + (0.17605714039701423 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13226191911708354 = (14.854376597342942 - 12.88970824129047) / 14.854376597342942 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in GuardedDevirtualization.ThreeClassInterface

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Call - Duration of single invocation 2.23 ns 0.50 ns 0.22 0.10 True
Call - Duration of single invocation 1.96 ns 0.50 ns 0.25 0.12 True
Call - Duration of single invocation 7.85 ns 1.70 ns 0.22 0.03 True
Call - Duration of single invocation 7.86 ns 1.71 ns 0.22 0.03 True
Call - Duration of single invocation 7.54 ns 1.57 ns 0.21 0.03 True
Call - Duration of single invocation 7.55 ns 1.59 ns 0.21 0.02 True
Call - Duration of single invocation 7.55 ns 1.65 ns 0.22 0.01 True
Call - Duration of single invocation 9.04 ns 5.09 ns 0.56 0.03 True
Call - Duration of single invocation 2.24 ns 0.49 ns 0.22 0.10 True
Call - Duration of single invocation 7.85 ns 1.70 ns 0.22 0.03 True

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'GuardedDevirtualization.ThreeClassInterface*'

Payloads

Baseline
Compare

Histogram

GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.00 pD=1.00)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.4951728220550588 < 1.8601058959418515.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 199.20618401547026 (T) = (0 -0.49226153578400667) / Math.Sqrt((0.01922301130066395 / (299)) + (1.7585476800729875E-06 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7646043965531707 = (2.091209557765583 - 0.49226153578400667) / 2.091209557765583 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=1.00 pD=0.00)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.49561315350418683 < 1.8603835139999958.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 188.4507868475652 (T) = (0 -0.4952891825421446) / Math.Sqrt((0.019931327131795972 / (299)) + (3.941172749936408E-06 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.756901448752026 = (2.037400798974414 - 0.4952891825421446) / 2.037400798974414 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.33 pD=0.67)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.695226646318905 < 7.458407968207897.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 76.54829158454423 (T) = (0 -1.824739483809311) / Math.Sqrt((0.00145059158095576 / (299)) + (0.08058489082450672 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7676660466548576 = (7.853950993975381 - 1.824739483809311) / 7.853950993975381 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.33 pD=0.00)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.7081336305075092 < 7.462111046785598.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 66.05796725875729 (T) = (0 -1.8391310943011179) / Math.Sqrt((0.0011366671606589088 / (299)) + (0.10775482128470044 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7658538036556001 = (7.854627250045032 - 1.8391310943011179) / 7.854627250045032 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.00 pD=0.67)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.569166639870234 < 7.168788806520315.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 79.2661978618548 (T) = (0 -1.7305014683633704) / Math.Sqrt((0.011334680901069438 / (299)) + (0.06766043914750393 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7683339658532801 = (7.469810905760144 - 1.7305014683633704) / 7.469810905760144 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.67 pD=0.33)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.5894485740291262 < 7.172332294942829.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 82.3747008729793 (T) = (0 -1.6122394072243666) / Math.Sqrt((0.011643889224553252 / (299)) + (0.06525799604240308 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7842047489570767 = (7.47115332442455 - 1.6122394072243666) / 7.47115332442455 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.67 pD=0.00)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.6507321692535106 < 7.172867575561544.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 285.2412472366761 (T) = (0 -1.4099838290409932) / Math.Sqrt((0.011250096796172343 / (299)) + (0.0053790223993768135 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.8112533964546055 = (7.4702474246212605 - 1.4099838290409932) / 7.4702474246212605 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.33 pD=0.33)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.087448035384783 < 8.41083154222107.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 38.98787284652369 (T) = (0 -5.311106151721412) / Math.Sqrt((0.02661649558514795 / (299)) + (0.08749312154488521 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.37741101378270014 = (8.530677974228887 - 5.311106151721412) / 8.530677974228887 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.00 pD=0.00)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.49074761100886116 < 1.9910766290740451.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 196.3823653809946 (T) = (0 -0.4912964192074012) / Math.Sqrt((0.01972627260947229 / (299)) + (2.317059629590464E-06 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7647673186379934 = (2.0885551121671337 - 0.4912964192074012) / 2.0885551121671337 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.00 pD=0.33)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.6984825303819446 < 7.459205954451746.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 61.34899246049985 (T) = (0 -1.9093959235711702) / Math.Sqrt((0.0002456551061293562 / (299)) + (0.12199734914888043 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7568493360813042 = (7.852727575564545 - 1.9093959235711702) / 7.852727575564545 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Depth

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadSpanEmptyLoop - Duration of single invocation 2.62 μs 1.81 μs 0.69 0.07 False
ReadSpanEmptyLoop - Duration of single invocation 88.54 ns 61.50 ns 0.69 0.17 False
ReadSpanEmptyLoop - Duration of single invocation 22.72 μs 17.85 μs 0.79 0.03 False
ReadSpanEmptyLoop - Duration of single invocation 2.69 μs 1.76 μs 0.65 0.14 True

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Depth*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Depth.ReadSpanEmptyLoop(Depth: 64)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.812902014013416 < 2.477706141403535.
IsChangePoint: Marked as a change because one of 3/8/2023 10:34:21 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 86.05227388124459 (T) = (0 -1768.0726121944406) / Math.Sqrt((3680.8333088872428 / (299)) + (989.0272788606935 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.31392910907012195 = (2577.099007652187 - 1768.0726121944406) / 2577.099007652187 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Depth.ReadSpanEmptyLoop(Depth: 1)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 61.49880337687462 < 84.21947743490352.
IsChangePoint: Marked as a change because one of 3/10/2023 11:41:11 AM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 15.104911080998956 (T) = (0 -63.84288384691843) / Math.Sqrt((38.5643354430188 / (299)) + (12.178123968038946 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19630302818893808 = (79.43651162832428 - 63.84288384691843) / 79.43651162832428 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Depth.ReadSpanEmptyLoop(Depth: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.848785714285714 < 21.632379278543777.
IsChangePoint: Marked as a change because one of 3/7/2023 10:25:17 PM, 4/25/2023 1:57:48 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 9.018046866066273 (T) = (0 -18943.20320991205) / Math.Sqrt((241982.3853363935 / (299)) + (1343894.282287652 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13319414870574028 = (21854.032459087877 - 18943.20320991205) / 21854.032459087877 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Depth.ReadSpanEmptyLoop(Depth: 65)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.7553712259401104 < 2.4623930586604397.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 41.377538506043955 (T) = (0 -1847.5172317047525) / Math.Sqrt((6744.954079148312 / (299)) + (4691.864767799268 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3048669676709167 = (2657.7894385403315 - 1847.5172317047525) / 2657.7894385403315 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Sort<BigStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LinqQuery - Duration of single invocation 30.44 μs 16.59 μs 0.55 0.13 True
Array_ComparerClass - Duration of single invocation 25.37 μs 11.84 μs 0.47 0.36 False
LinqOrderByExtension - Duration of single invocation 30.10 μs 16.52 μs 0.55 0.12 True
Array_Comparison - Duration of single invocation 20.63 μs 12.65 μs 0.61 0.38 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;BigStruct&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<BigStruct>.LinqQuery(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.591704 < 28.893127733333337.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 163.11457469357214 (T) = (0 -16830.04978021978) / Math.Sqrt((941369.8751523645 / (299)) + (56565.51579291047 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4563199522989539 = (30955.798086366663 - 16830.04978021978) / 30955.798086366663 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<BigStruct>.Array_ComparerClass(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.84439333333333 < 19.331107738461537.
IsChangePoint: Marked as a change because one of 4/25/2023 9:59:52 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 65.47310807607745 (T) = (0 -12048.681224006763) / Math.Sqrt((2035054.4562905326 / (299)) + (198986.2069238253 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.44692216596342 = (21784.784134397032 - 12048.681224006763) / 21784.784134397032 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<BigStruct>.LinqOrderByExtension(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.523477142857143 < 28.770427.
IsChangePoint: Marked as a change because one of 3/25/2023 3:38:32 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 58.17052199885116 (T) = (0 -17172.18650549451) / Math.Sqrt((978537.3111065048 / (299)) + (684249.7743471022 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.44474103104581086 = (30926.445974997434 - 17172.18650549451) / 30926.445974997434 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<BigStruct>.Array_Comparison(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.645855714285718 < 20.07012053333333.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 48.40941132429983 (T) = (0 -12703.00878050155) / Math.Sqrt((1961543.9199694416 / (299)) + (380321.2598871975 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.41901278993676094 = (21864.52396967372 - 12703.00878050155) / 21864.52396967372 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryGetValueFalse<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 267.71 μs 242.57 μs 0.91 0.02 False
FrozenDictionary - Duration of single invocation 8.06 μs 7.36 μs 0.91 0.01 False
SortedDictionary - Duration of single invocation 312.97 μs 255.72 μs 0.82 0.08 False
FrozenDictionaryOptimized - Duration of single invocation 7.25 μs 5.95 μs 0.82 0.01 True

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<String, String>.SortedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 242.56735897435897 < 290.66747547055684.
IsChangePoint: Marked as a change because one of 5/11/2023 5:02:01 PM, 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 43.87188288491697 (T) = (0 -242403.37044442177) / Math.Sqrt((55783702.580994375 / (299)) + (6720838.367472231 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13180027906935016 = (279202.3132472125 - 242403.37044442177) / 279202.3132472125 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.TryGetValueFalse<String, String>.FrozenDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.35942102370859 < 7.647458093009051.
IsChangePoint: Marked as a change because one of 5/16/2023 12:30:38 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 34.86290699526373 (T) = (0 -7398.723328986491) / Math.Sqrt((13773.13657975598 / (264)) + (3473.10626478979 / (13))) is greater than 1.9686278708474185 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (264) + (13) - 2, .975) and 0.07766368835624862 = (8021.719665141209 - 7398.723328986491) / 8021.719665141209 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.TryGetValueFalse<String, String>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 255.71588797814206 < 315.90669140624993.
IsChangePoint: Marked as a change because one of 3/6/2023 11:36:59 AM, 3/14/2023 8:42:09 PM, 4/26/2023 4:28:25 PM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 13.45690318319228 (T) = (0 -269512.0411643494) / Math.Sqrt((182447766.70688203 / (299)) + (153464602.06825086 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.10970979666803217 = (302723.80865888833 - 269512.0411643494) / 302723.80865888833 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.TryGetValueFalse<String, String>.FrozenDictionaryOptimized(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.947725919913419 < 6.8896224609585435.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 20.67515680161135 (T) = (0 -5998.683273955612) / Math.Sqrt((992.9490495848167 / (262)) + (49755.509141587616 / (13))) is greater than 1.9686916197951025 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (262) + (13) - 2, .975) and 0.17582350415466316 = (7278.396440804728 - 5998.683273955612) / 7278.396440804728 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Sort<IntStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LinqQuery - Duration of single invocation 23.99 μs 13.74 μs 0.57 0.04 True
Array_ComparerClass - Duration of single invocation 18.36 μs 10.57 μs 0.58 0.10 True
LinqOrderByExtension - Duration of single invocation 25.71 μs 13.82 μs 0.54 0.02 True
Array_Comparison - Duration of single invocation 17.51 μs 10.60 μs 0.61 0.10 True

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;IntStruct&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<IntStruct>.LinqQuery(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.742238666666665 < 23.896839599999996.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 136.85143718999325 (T) = (0 -13834.124043956046) / Math.Sqrt((516038.88426448096 / (299)) + (78779.28866135393 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.46606041349003996 = (25909.530578883172 - 13834.124043956046) / 25909.530578883172 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<IntStruct>.Array_ComparerClass(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.568150666666666 < 17.170533846153848.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 74.45878636132296 (T) = (0 -10021.049650042265) / Math.Sqrt((474409.93566711375 / (299)) + (167989.20287619287 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4722937772664981 = (18989.8265708021 - 10021.049650042265) / 18989.8265708021 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<IntStruct>.LinqOrderByExtension(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.81808307692308 < 23.74549433095238.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 134.4528796858403 (T) = (0 -13784.136816004506) / Math.Sqrt((452083.0748344718 / (299)) + (85575.09502258372 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4674016703188877 = (25880.92385542706 - 13784.136816004506) / 25880.92385542706 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<IntStruct>.Array_Comparison(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.602801428571428 < 16.991062233333334.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 64.5592105882363 (T) = (0 -10590.70778078332) / Math.Sqrt((713642.2103186423 / (299)) + (182118.61738574636 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4383757629844781 = (18857.284074958145 - 10590.70778078332) / 18857.284074958145 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.DependencyInjection.GetService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ServiceScopeProvider - Duration of single invocation 24.94 ns 14.63 ns 0.59 0.21 False
EmptyEnumerable - Duration of single invocation 24.15 ns 13.86 ns 0.57 0.31 False
Transient - Duration of single invocation 26.15 ns 16.54 ns 0.63 0.15 False
Singleton - Duration of single invocation 20.47 ns 11.22 ns 0.55 0.17 False
Scoped - Duration of single invocation 50.58 ns 30.40 ns 0.60 0.13 True
ServiceScope - Duration of single invocation 50.88 ns 30.08 ns 0.59 0.07 True

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetService*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.GetService.ServiceScopeProvider

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.625824086982679 < 22.476528904157753.
IsChangePoint: Marked as a change because one of 3/24/2023 7:28:10 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 98.38756135110935 (T) = (0 -14.920969176177639) / Math.Sqrt((1.360242407480117 / (299)) + (0.07878531623855606 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.40447684636807 = (25.055229314223638 - 14.920969176177639) / 25.055229314223638 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.GetService.EmptyEnumerable

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.85679558658581 < 23.403975230676455.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 36.47845067619191 (T) = (0 -14.042682407106982) / Math.Sqrt((3.7492570387235267 / (299)) + (0.8003317783227476 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4142247895126466 = (23.972817824475275 - 14.042682407106982) / 23.972817824475275 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.GetService.Transient

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.54071640641483 < 25.562346881021774.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 77.78004015893231 (T) = (0 -16.17069781826692) / Math.Sqrt((1.3666259149803481 / (299)) + (0.1980883083571038 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.40368235063368413 = (27.117590491327746 - 16.17069781826692) / 27.117590491327746 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.GetService.Singleton

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.220132350808331 < 19.65381611162645.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 125.66676239551877 (T) = (0 -11.29600675015865) / Math.Sqrt((1.2233898783683257 / (299)) + (0.021617543202927237 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.45767582667369894 = (20.828882992390906 - 11.29600675015865) / 20.828882992390906 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.GetService.Scoped

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 30.402762185524487 < 48.42624328085123.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 106.23781447057051 (T) = (0 -29.793535565936683) / Math.Sqrt((4.10292817500683 / (299)) + (0.41106685649982344 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.43159061370531987 = (52.41562909464489 - 29.793535565936683) / 52.41562909464489 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.GetService.ServiceScope

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 30.075252285584927 < 49.33636958763204.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 133.51263292203222 (T) = (0 -30.033432920409066) / Math.Sqrt((5.164141098974368 / (299)) + (0.10640094225640813 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.41495571040820517 = (51.335315043181446 - 30.033432920409066) / 51.335315043181446 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in JetStream.Poker

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Play - Duration of single invocation 8.89 ms 7.99 ms 0.90 0.05 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'JetStream.Poker*'

Payloads

Baseline
Compare

Histogram

JetStream.Poker.Play

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.985909247311828 < 8.44741719047619.
IsChangePoint: Marked as a change because one of 3/28/2023 8:13:28 AM, 4/10/2023 1:33:42 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 25.458386358885154 (T) = (0 -8077923.537279923) / Math.Sqrt((64487182315.99347 / (299)) + (10830190711.294249 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.092611336989274 = (8902385.346624544 - 8077923.537279923) / 8902385.346624544 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 3.72 ns 2.22 ns 0.60 0.05 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.ToString(value: 0)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.216899272866904 < 3.5363307459398374.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 124.55139087513525 (T) = (0 -2.2233125733096935) / Math.Sqrt((0.020469406402451082 / (299)) + (0.0010706299911086364 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4075734343198038 = (3.752891416604505 - 2.2233125733096935) / 3.752891416604505 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToString - Duration of single invocation 198.45 ns 163.87 ns 0.83 0.06 False
SerializeToStream - Duration of single invocation 233.56 ns 189.43 ns 0.81 0.10 False
SerializeToString - Duration of single invocation 264.03 ns 206.73 ns 0.78 0.11 False
SerializeToWriter - Duration of single invocation 107.38 ns 87.68 ns 0.82 0.16 False
SerializeToUtf8Bytes - Duration of single invocation 172.50 ns 139.97 ns 0.81 0.12 False
SerializeToStream - Duration of single invocation 334.66 ns 247.30 ns 0.74 0.17 False
SerializeToUtf8Bytes - Duration of single invocation 237.93 ns 181.36 ns 0.76 0.14 False
SerializeObjectProperty - Duration of single invocation 426.94 ns 311.01 ns 0.73 0.11 True
SerializeObjectProperty - Duration of single invocation 485.67 ns 348.69 ns 0.72 0.10 True
SerializeToWriter - Duration of single invocation 186.97 ns 125.01 ns 0.67 0.17 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;SimpleStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 163.87197745460801 < 192.04601675825856.
IsChangePoint: Marked as a change because one of 4/19/2023 4:18:46 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 37.545715620094036 (T) = (0 -163.86637644795107) / Math.Sqrt((46.706215981782734 / (299)) + (11.266144310463192 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18813058816454498 = (201.83834254512172 - 163.86637644795107) / 201.83834254512172 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 189.43379099232828 < 226.69807015070427.
IsChangePoint: Marked as a change because one of 4/18/2023 7:36:21 AM, 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 42.92863035392227 (T) = (0 -188.52885303649808) / Math.Sqrt((121.29488777920102 / (299)) + (14.177426988856821 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2178509015477947 = (241.0395325003606 - 188.52885303649808) / 241.0395325003606 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 206.72845169184237 < 256.0617290589215.
IsChangePoint: Marked as a change because one of 3/25/2023 2:21:39 PM, 4/17/2023 1:54:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 83.46961877142958 (T) = (0 -203.8299042182625) / Math.Sqrt((118.42262422403795 / (299)) + (3.3525178396361803 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24877307502299692 = (271.3293379686867 - 203.8299042182625) / 271.3293379686867 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToWriter(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 87.67735208143176 < 103.95245693907349.
IsChangePoint: Marked as a change because one of 3/21/2023 12:39:01 PM, 4/19/2023 3:30:01 PM, 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 14.501459363096012 (T) = (0 -86.83673825639387) / Math.Sqrt((24.74233034485402 / (299)) + (23.133978646613084 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1855969779205773 = (106.62624757294347 - 86.83673825639387) / 106.62624757294347 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 139.96676430664732 < 168.8602636016127.
IsChangePoint: Marked as a change because one of 4/17/2023 7:48:00 AM, 5/6/2023 11:12:53 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 17.827491755117865 (T) = (0 -139.66323275424244) / Math.Sqrt((57.857281871886535 / (299)) + (63.589354201650586 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22350687249726547 = (179.864093843832 - 139.66323275424244) / 179.864093843832 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 247.30474237793607 < 322.9188158183426.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 46.94618521293471 (T) = (0 -246.9670417049416) / Math.Sqrt((122.12993208651763 / (299)) + (46.885715431171 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2758327951293926 = (341.0359376175688 - 246.9670417049416) / 341.0359376175688 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 181.35731649987468 < 231.64849404878916.
IsChangePoint: Marked as a change because one of 3/20/2023 4:51:16 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 75.94630152653963 (T) = (0 -182.1074439399416) / Math.Sqrt((96.14611116656675 / (299)) + (6.769176282588354 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27679796244638133 = (251.8071499853041 - 182.1074439399416) / 251.8071499853041 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeObjectProperty(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 311.0117897665555 < 407.3771046195736.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 58.11111560578486 (T) = (0 -316.6126612799831) / Math.Sqrt((167.51947718609634 / (299)) + (46.6393485840943 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27209483315627364 = (434.9641625060158 - 316.6126612799831) / 434.9641625060158 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeObjectProperty(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 348.6898929862209 < 470.9392640615781.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 100.20798814837032 (T) = (0 -349.04411273244557) / Math.Sqrt((184.92079589317797 / (299)) + (19.752138786507555 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.29566066427563875 = (495.5624299663448 - 349.04411273244557) / 495.5624299663448 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToWriter(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 125.00912261199956 < 176.53534040632113.
IsChangePoint: Marked as a change because one of 3/20/2023 10:25:17 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 44.28444569137907 (T) = (0 -122.56016276442269) / Math.Sqrt((126.83320760299127 / (299)) + (23.718748483397764 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.35142300738459437 = (188.96779281391906 - 122.56016276442269) / 188.96779281391906 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Reflection.Invoke

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Property_Set_class - Duration of single invocation 37.26 ns 31.19 ns 0.84 0.08 False
Property_Set_int - Duration of single invocation 40.85 ns 35.10 ns 0.86 0.13 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Invoke.Property_Set_class

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 31.18660275602641 < 36.05548215060764.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.787948268295144 (T) = (0 -31.205811893877062) / Math.Sqrt((11.591281597044135 / (299)) + (0.17797260196823853 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08580382096390078 = (34.134699542038696 - 31.205811893877062) / 34.134699542038696 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Invoke.Property_Set_int

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 35.10111080383481 < 38.644210139654824.
IsChangePoint: Marked as a change because one of 4/26/2023 8:04:03 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 11.73227491196404 (T) = (0 -35.56308152662619) / Math.Sqrt((7.774589341195823 / (299)) + (0.6129254315121334 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08191668108148763 = (38.73622447308915 - 35.56308152662619) / 38.73622447308915 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromString - Duration of single invocation 523.28 ns 384.67 ns 0.74 0.05 True
DeserializeFromReader - Duration of single invocation 701.17 ns 535.42 ns 0.76 0.05 True
DeserializeFromStream - Duration of single invocation 811.88 ns 598.55 ns 0.74 0.06 True
DeserializeFromStream - Duration of single invocation 815.54 ns 603.02 ns 0.74 0.05 True
DeserializeFromUtf8Bytes - Duration of single invocation 456.88 ns 341.83 ns 0.75 0.06 True
DeserializeFromReader - Duration of single invocation 724.03 ns 526.90 ns 0.73 0.08 True
DeserializeFromString - Duration of single invocation 519.47 ns 387.08 ns 0.75 0.07 True
DeserializeFromUtf8Bytes - Duration of single invocation 441.45 ns 344.18 ns 0.78 0.06 True

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;LoginViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 384.66958137543156 < 495.2442221547437.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 80.31485528977504 (T) = (0 -387.4692900076774) / Math.Sqrt((170.65626910580366 / (299)) + (22.147367895371257 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2381545726545115 = (508.59305063724474 - 387.4692900076774) / 508.59305063724474 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromReader(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 535.4230528118784 < 668.1867842245993.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 68.5689857365279 (T) = (0 -538.6167645315558) / Math.Sqrt((241.7774129983133 / (299)) + (61.628396964003734 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2307056269628748 = (700.1439025286654 - 538.6167645315558) / 700.1439025286654 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 598.5526511848656 < 783.0826323785913.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 124.64587533742785 (T) = (0 -599.1900902535097) / Math.Sqrt((397.037002197924 / (299)) + (22.000422822588163 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2655269791558009 = (815.8095304369438 - 599.1900902535097) / 815.8095304369438 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 603.0157817871817 < 779.0170078448994.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 87.7147272964874 (T) = (0 -597.7598409437192) / Math.Sqrt((265.7019836819076 / (299)) + (62.74159634724594 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2596939608283306 = (807.4496347653119 - 597.7598409437192) / 807.4496347653119 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 341.82637827776813 < 430.163329434282.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 49.42791501261821 (T) = (0 -342.3453872787353) / Math.Sqrt((107.00760612640302 / (299)) + (50.63472729196527 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22943425381929428 = (444.27797235416136 - 342.3453872787353) / 444.27797235416136 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromReader(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 526.8970136872124 < 674.6232550601604.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 63.35840579073349 (T) = (0 -541.9018653033542) / Math.Sqrt((354.3652199651464 / (299)) + (77.97813098603257 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23859734162555846 = (711.715226290763 - 541.9018653033542) / 711.715226290763 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 387.0784703158857 < 493.5794463488224.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 51.67520507668352 (T) = (0 -390.4137972499776) / Math.Sqrt((237.1276379031254 / (299)) + (63.04399327444731 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23920263177094733 = (513.163969216618 - 390.4137972499776) / 513.163969216618 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 344.18012354114666 < 428.29149114526894.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 29.89744790676927 (T) = (0 -339.79170240822776) / Math.Sqrt((153.5378137945909 / (299)) + (143.36218131313208 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23012758741782496 = (441.36105782587566 - 339.79170240822776) / 441.36105782587566 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToUtf8Bytes - Duration of single invocation 288.90 μs 252.05 μs 0.87 0.10 False
SerializeObjectProperty - Duration of single invocation 310.89 μs 282.51 μs 0.91 0.10 False
SerializeToWriter - Duration of single invocation 286.39 μs 249.99 μs 0.87 0.12 False
SerializeToString - Duration of single invocation 370.30 μs 296.45 μs 0.80 0.08 True
SerializeToUtf8Bytes - Duration of single invocation 348.60 μs 279.55 μs 0.80 0.11 True
SerializeObjectProperty - Duration of single invocation 372.65 μs 297.28 μs 0.80 0.09 False
SerializeToString - Duration of single invocation 309.59 μs 272.83 μs 0.88 0.15 False
SerializeToWriter - Duration of single invocation 350.23 μs 271.36 μs 0.77 0.10 True
SerializeToStream - Duration of single invocation 347.25 μs 279.77 μs 0.81 0.09 False
SerializeToStream - Duration of single invocation 346.38 μs 275.25 μs 0.79 0.10 True

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 252.0522189349113 < 280.80284366576825.
IsChangePoint: Marked as a change because one of 4/18/2023 7:42:00 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 38.35063674017662 (T) = (0 -250683.36107001978) / Math.Sqrt((118241200.15484786 / (299)) + (25019687.19401968 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1889839407072867 = (309097.9003407659 - 250683.36107001978) / 309097.9003407659 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeObjectProperty(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 282.50713848039214 < 300.18582304621845.
IsChangePoint: Marked as a change because one of 4/30/2023 4:58:26 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 22.75625450042409 (T) = (0 -277938.8686350493) / Math.Sqrt((136263421.98484793 / (299)) + (56835698.60771095 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15246781099303786 = (327939.01192201936 - 277938.8686350493) / 327939.01192201936 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToWriter(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 249.99254807692313 < 278.2491521545584.
IsChangePoint: Marked as a change because one of 4/17/2023 7:48:00 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 37.42489085998283 (T) = (0 -246912.92711638307) / Math.Sqrt((113965628.88157293 / (299)) + (27358200.49430263 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1928750251250078 = (305916.59879515565 - 246912.92711638307) / 305916.59879515565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 296.44738881401616 < 353.3826114469752.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 60.69853214741775 (T) = (0 -289471.38872169895) / Math.Sqrt((106680073.76510316 / (299)) + (25843684.084609594 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.243047371863254 = (382416.78271761676 - 289471.38872169895) / 382416.78271761676 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 279.548652742347 < 336.8185774373196.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 53.123765843149805 (T) = (0 -273430.92011555925) / Math.Sqrt((101525915.52917024 / (299)) + (34237773.20976276 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2509408370838232 = (365032.4749397097 - 273430.92011555925) / 365032.4749397097 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeObjectProperty(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 297.27759433962257 < 355.2997309604558.
IsChangePoint: Marked as a change because one of 4/30/2023 6:17:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 50.74541203809596 (T) = (0 -290934.223980653) / Math.Sqrt((127477321.64776953 / (299)) + (37895200.02641031 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2417540532728858 = (383693.7411092519 - 290934.223980653) / 383693.7411092519 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 272.82717524509815 < 301.1318430473856.
IsChangePoint: Marked as a change because one of 4/20/2023 12:36:45 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 45.40811355998702 (T) = (0 -274129.6117087995) / Math.Sqrt((132991729.25756294 / (299)) + (11925180.971415818 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16200394010108007 = (327125.17973159126 - 274129.6117087995) / 327125.17973159126 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToWriter(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 271.3628771551724 < 334.5399981303419.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 67.41287775119503 (T) = (0 -266430.0733813632) / Math.Sqrt((104592324.46867111 / (299)) + (23442088.597669393 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27074732548133335 = (365346.7209526749 - 266430.0733813632) / 365346.7209526749 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 279.7662587412587 < 331.9322854662699.
IsChangePoint: Marked as a change because one of 4/19/2023 1:39:34 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 56.782991676545016 (T) = (0 -274942.2740720817) / Math.Sqrt((115926881.79172812 / (299)) + (26217227.893546496 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24256483709200524 = (362991.16747696965 - 274942.2740720817) / 362991.16747696965 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 275.2493589743589 < 332.501397040344.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 66.38140875392352 (T) = (0 -273140.31919685774) / Math.Sqrt((99203665.99455102 / (299)) + (21528986.087601032 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2552055245413316 = (366732.4721073544 - 273140.31919685774) / 366732.4721073544 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NewDefaultOptions - Duration of single invocation 1.20 μs 844.19 ns 0.70 0.01 True
NewCustomConverter - Duration of single invocation 25.77 μs 21.46 μs 0.83 0.01 False
NewCachedCustomConverter - Duration of single invocation 1.39 μs 976.83 ns 0.70 0.01 True
NewCustomizedOptions - Duration of single invocation 1.31 μs 923.02 ns 0.70 0.01 True
CachedDefaultOptions - Duration of single invocation 763.50 ns 484.02 ns 0.63 0.03 True
CachedJsonSerializerContext - Duration of single invocation 681.53 ns 450.33 ns 0.66 0.07 True

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ColdStartSerialization&lt;SimpleStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewDefaultOptions

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 844.1912151627364 < 1.159431625903131.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 163.10489010706414 (T) = (0 -840.7426283105871) / Math.Sqrt((506.71717561822965 / (299)) + (44.74791815165492 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3054089210379724 = (1210.413801408108 - 840.7426283105871) / 1210.413801408108 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCustomConverter

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.458504166666668 < 24.537687623706788.
IsChangePoint: Marked as a change because one of 4/25/2023 1:57:48 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 23.48174761822775 (T) = (0 -21783.47224441836) / Math.Sqrt((1353480.1504037185 / (299)) + (68326.52204652216 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09634568049310753 = (24105.979215929827 - 21783.47224441836) / 24105.979215929827 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCachedCustomConverter

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 976.8308946677531 < 1.3256673159721117.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 113.65394106899501 (T) = (0 -983.7062185445945) / Math.Sqrt((694.5785581436011 / (299)) + (129.048446126429 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2879392102185598 = (1381.4919072380508 - 983.7062185445945) / 1381.4919072380508 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCustomizedOptions

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 923.0182803332651 < 1.2463477056694674.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 129.74362465810538 (T) = (0 -923.3359426057468) / Math.Sqrt((725.0382201720457 / (299)) + (78.84372403567235 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2904908579534047 = (1301.3728617257323 - 923.3359426057468) / 1301.3728617257323 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.CachedDefaultOptions

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 484.01779850512884 < 734.27674464219.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 98.48866507935261 (T) = (0 -492.36068454166) / Math.Sqrt((430.42382810763485 / (299)) + (89.21794608331054 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3656338378291346 = (776.1458821459704 - 492.36068454166) / 776.1458821459704 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.CachedJsonSerializerContext

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 450.333134662621 < 663.9313591035284.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 118.39348937508966 (T) = (0 -458.3160733399129) / Math.Sqrt((490.36641991597224 / (299)) + (26.215014085059362 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3306417878276913 = (684.7097189597661 - 458.3160733399129) / 684.7097189597661 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<StructRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToWriter - Duration of single invocation 370.71 ns 248.03 ns 0.67 0.17 False
SerializeToWriter - Duration of single invocation 210.98 ns 190.52 ns 0.90 0.08 False
SerializeToString - Duration of single invocation 326.13 ns 277.75 ns 0.85 0.09 False
SerializeObjectProperty - Duration of single invocation 548.16 ns 422.61 ns 0.77 0.04 True
SerializeObjectProperty - Duration of single invocation 692.79 ns 535.67 ns 0.77 0.08 True
SerializeToUtf8Bytes - Duration of single invocation 454.78 ns 327.59 ns 0.72 0.10 True
SerializeToString - Duration of single invocation 469.74 ns 359.79 ns 0.77 0.10 False
SerializeToStream - Duration of single invocation 556.91 ns 396.57 ns 0.71 0.10 True
SerializeToUtf8Bytes - Duration of single invocation 279.45 ns 248.77 ns 0.89 0.08 False
SerializeToStream - Duration of single invocation 338.57 ns 290.87 ns 0.86 0.05 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;StructRecord&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToWriter(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 248.03028003238109 < 351.7454186889524.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 47.45640219325491 (T) = (0 -263.25045471791213) / Math.Sqrt((224.50419635434434 / (299)) + (59.00593101231837 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.29309339378449284 = (372.3977855112274 - 263.25045471791213) / 372.3977855112274 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToWriter(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 190.52138404699875 < 199.31916956481922.
IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 27.077810068916456 (T) = (0 -183.53658290199516) / Math.Sqrt((168.51668507900445 / (299)) + (11.413744806834908 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1504815585205576 = (216.04779124319526 - 183.53658290199516) / 216.04779124319526 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 277.7501239776188 < 310.95071680418164.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/4/2023 10:11:14 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 14.361461998611219 (T) = (0 -282.2164506553085) / Math.Sqrt((178.64344236798902 / (299)) + (129.98430183551582 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14210989923620063 = (328.96573862321605 - 282.2164506553085) / 328.96573862321605 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeObjectProperty(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 422.6127293541515 < 533.0717637628535.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 117.34253073640163 (T) = (0 -424.31390406651485) / Math.Sqrt((234.66884819124425 / (299)) + (8.792914258330322 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2505391014388272 = (566.1588281404935 - 424.31390406651485) / 566.1588281404935 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeObjectProperty(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 535.6724135882652 < 668.3038328575196.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 48.26908498904855 (T) = (0 -502.6852249772134) / Math.Sqrt((366.12818396893243 / (299)) + (218.28851287716847 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2895553404064779 = (707.5642250091972 - 502.6852249772134) / 707.5642250091972 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 327.5876345316216 < 427.09202809228225.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 54.757955437274624 (T) = (0 -325.512004095906) / Math.Sqrt((201.00081168350403 / (299)) + (55.331494391381746 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2719090994832372 = (447.0760503460127 - 325.512004095906) / 447.0760503460127 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 359.7856763998615 < 457.24038562495963.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 69.14543216727033 (T) = (0 -355.69609255856943) / Math.Sqrt((312.5871540813622 / (299)) + (28.53852911055999 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25923125717691314 = (480.17157311876224 - 355.69609255856943) / 480.17157311876224 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 396.5700782517323 < 527.9239623658765.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 82.98961362786602 (T) = (0 -391.3276993398777) / Math.Sqrt((257.51203644385873 / (299)) + (30.953726658673926 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27634026562410324 = (540.7620194280521 - 391.3276993398777) / 540.7620194280521 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 248.76521621131397 < 273.64867830722284.
IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 22.788332510939426 (T) = (0 -250.3715926732786) / Math.Sqrt((166.14611497687324 / (299)) + (46.024950867661914 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15555430352606628 = (296.49223593503984 - 250.3715926732786) / 296.49223593503984 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 290.8743825006041 < 337.70163805503483.
IsChangePoint: Marked as a change because one of 4/17/2023 1:30:24 AM, 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 40.556404430991265 (T) = (0 -294.0654103324448) / Math.Sqrt((115.78996934496723 / (299)) + (23.328790842071673 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16923790240834705 = (353.9706628226408 - 294.0654103324448) / 353.9706628226408 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetFileNameWithoutExtension - Duration of single invocation 30.59 ns 23.29 ns 0.76 0.09 False
GetFileName - Duration of single invocation 28.80 ns 22.14 ns 0.77 0.08 False
GetPathRoot - Duration of single invocation 28.54 ns 25.69 ns 0.90 0.06 False
GetTempPath - Duration of single invocation 320.56 ns 282.65 ns 0.88 0.06 False
Combine - Duration of single invocation 3.90 ns 1.95 ns 0.50 0.09 False

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.GetFileNameWithoutExtension

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.285939075798584 < 30.24363099489407.
IsChangePoint: Marked as a change because one of 3/17/2023 9:25:32 PM, 3/20/2023 8:31:51 AM, 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 24.563548940064482 (T) = (0 -23.851801643951305) / Math.Sqrt((3.555275972193092 / (299)) + (0.5763445326847104 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19626634872136972 = (29.676251088910302 - 23.851801643951305) / 29.676251088910302 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_Path.GetFileName

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 22.141979509564674 < 26.891045535122455.
IsChangePoint: Marked as a change because one of 3/10/2023 12:53:36 AM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 19.957316822286973 (T) = (0 -22.36951469287022) / Math.Sqrt((5.953987275595339 / (299)) + (0.07380231243057551 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12489417201956322 = (25.562068012384778 - 22.36951469287022) / 25.562068012384778 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_Path.GetPathRoot

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 25.69047405561329 < 26.99507897943935.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/9/2023 11:04:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 11.091443351022045 (T) = (0 -23.707443334651153) / Math.Sqrt((3.3657235835587516 / (299)) + (1.7770140203890115 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1525091282840883 = (27.973685765665863 - 23.707443334651153) / 27.973685765665863 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_Path.GetTempPath

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 282.65464974304774 < 297.58474530689.
IsChangePoint: Marked as a change because one of 3/6/2023 4:48:38 PM, 3/14/2023 8:42:09 PM, 3/20/2023 8:31:51 AM, 4/26/2023 6:27:29 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 21.01216870092628 (T) = (0 -284.5532100378051) / Math.Sqrt((101.34756187849543 / (299)) + (8.557730589069225 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06867651228491549 = (305.53638321302293 - 284.5532100378051) / 305.53638321302293 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_Path.Combine

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.9499000657014156 < 3.703940274000561.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 42.72620442368472 (T) = (0 -1.936384590873489) / Math.Sqrt((0.24003328095412943 / (299)) + (0.0008627764650213386 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3941251043951313 = (3.1960139047192575 - 1.936384590873489) / 3.1960139047192575 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IndexerSetReverse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IList - Duration of single invocation 1.15 μs 1.02 μs 0.89 0.08 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSetReverse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSetReverse<Int32>.IList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.017549461803286 < 1.0917086506659497.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/25/2023 6:46:56 PM, 5/1/2023 10:17:01 AM, 5/5/2023 3:18:41 PM, 5/9/2023 1:22:51 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 46.12249438791928 (T) = (0 -1017.2359860431825) / Math.Sqrt((10164.217682545901 / (299)) + (0.8942286030720873 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2092519592509508 = (1286.422391991751 - 1017.2359860431825) / 1286.422391991751 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.Midpoint

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 402.33 ms 214.90 ms 0.53 0.47 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Midpoint*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.Midpoint.Test

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 214.89968947368422 < 398.69989865384616.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 49.10456736504381 (T) = (0 -221358457.44538864) / Math.Sqrt((382637276314956.1 / (299)) + (201546748979702.38 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4761085668474043 = (422527347.15154773 - 221358457.44538864) / 422527347.15154773 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
JsonNet_ - Duration of single invocation 39.41 μs 29.36 μs 0.74 0.03 True
SystemTextJson_SourceGen_ - Duration of single invocation 14.19 μs 12.16 μs 0.86 0.02 False
DataContractJsonSerializer_ - Duration of single invocation 84.10 μs 51.63 μs 0.61 0.32 False
SystemTextJson_Reflection_ - Duration of single invocation 14.39 μs 12.29 μs 0.85 0.06 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.JsonNet_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 29.360058699225167 < 37.46078593936064.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 64.76310000058184 (T) = (0 -28617.128904423033) / Math.Sqrt((459733.505686455 / (299)) + (325043.10860122927 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2693734233240084 = (39167.92766370141 - 28617.128904423033) / 39167.92766370141 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.SystemTextJson_SourceGen_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.160456194919647 < 13.285798347404745.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 46.93611218874988 (T) = (0 -12269.693127005912) / Math.Sqrt((176376.02323134796 / (299)) + (20607.271215952474 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15139594500615502 = (14458.67840814749 - 12269.693127005912) / 14458.67840814749 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.DataContractJsonSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 51.631436242402785 < 81.04848563583839.
IsChangePoint: Marked as a change because one of 4/4/2023 4:23:54 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 86.49367003447585 (T) = (0 -53248.671760046804) / Math.Sqrt((12857068.217568884 / (299)) + (1178972.6362103696 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.37261473341558754 = (84873.95958458069 - 53248.671760046804) / 84873.95958458069 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.SystemTextJson_Reflection_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.28655856602237 < 13.613466186382523.
IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 24.057385412887488 (T) = (0 -12195.054613137601) / Math.Sqrt((191620.41715409516 / (299)) + (121148.86428310332 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16449215516707874 = (14595.978587820775 - 12195.054613137601) / 14595.978587820775 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LinkedList - Duration of single invocation 6.12 μs 5.68 μs 0.93 0.02 False
ImmutableArray - Duration of single invocation 27.44 μs 24.58 μs 0.90 0.03 False
ImmutableSortedDictionary - Duration of single invocation 145.22 μs 102.23 μs 0.70 0.02 True
ImmutableSortedSet - Duration of single invocation 139.50 μs 97.82 μs 0.70 0.01 True
ConcurrentQueue - Duration of single invocation 5.59 μs 4.76 μs 0.85 0.02 True
SortedSet - Duration of single invocation 38.98 μs 26.54 μs 0.68 0.41 False
ImmutableHashSet - Duration of single invocation 195.48 μs 134.43 μs 0.69 0.05 True
ImmutableDictionary - Duration of single invocation 209.32 μs 164.22 μs 0.78 0.01 True
ConcurrentBag - Duration of single invocation 12.95 μs 9.77 μs 0.75 0.05 False
ICollection - Duration of single invocation 1.72 μs 1.48 μs 0.86 0.32 False
SortedDictionary - Duration of single invocation 48.72 μs 30.31 μs 0.62 0.05 True

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<Int32>.LinkedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.6771435314685315 < 5.810778304035814.
IsChangePoint: Marked as a change because one of 4/25/2023 1:57:48 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 8.062916444392098 (T) = (0 -5781.784802302253) / Math.Sqrt((5057.71797533212 / (299)) + (27963.031751235725 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06097196726491823 = (6157.201489993652 - 5781.784802302253) / 6157.201489993652 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CreateAddAndClear<Int32>.ImmutableArray(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 24.584203834115808 < 26.038707217261905.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 70.37034929548128 (T) = (0 -24650.27737589599) / Math.Sqrt((633462.3701249126 / (299)) + (9896.637163828764 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13284681425712264 = (28426.669913895847 - 24650.27737589599) / 28426.669913895847 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CreateAddAndClear<Int32>.ImmutableSortedDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 102.22530053688143 < 138.716599005141.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 48.25547145320826 (T) = (0 -106741.00637120887) / Math.Sqrt((3735487.2622025367 / (299)) + (8858504.726054983 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.273567365806309 = (146938.61666840836 - 106741.00637120887) / 146938.61666840836 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CreateAddAndClear<Int32>.ImmutableSortedSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 97.81808872767857 < 130.98858930546777.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 140.92483536074562 (T) = (0 -97672.31954165512) / Math.Sqrt((3696133.5462219003 / (299)) + (979677.5932999001 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2993938713147167 = (139411.16918994315 - 97672.31954165512) / 139411.16918994315 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CreateAddAndClear<Int32>.ConcurrentQueue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.761952704451346 < 5.294553410367366.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 9.978212151353697 (T) = (0 -4825.015742464872) / Math.Sqrt((1270.735066126613 / (299)) + (71822.00710537707 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13327784624753228 = (5566.969439485306 - 4825.015742464872) / 5566.969439485306 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CreateAddAndClear<Int32>.SortedSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 26.538565496471165 < 35.94370940611516.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 31.4106508468927 (T) = (0 -25818.978135073434) / Math.Sqrt((1991507.1518134642 / (299)) + (1495685.0836687817 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2979651638376213 = (36777.34608756876 - 25818.978135073434) / 36777.34608756876 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CreateAddAndClear<Int32>.ImmutableHashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 134.42748191978964 < 184.883664103836.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 127.5258981499706 (T) = (0 -129655.80298579276) / Math.Sqrt((15339376.149333106 / (299)) + (3197224.0451247175 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.34906147151475064 = (199182.86798525375 - 129655.80298579276) / 199182.86798525375 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CreateAddAndClear<Int32>.ImmutableDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 164.21880527210885 < 198.61676461805556.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 76.55531999492285 (T) = (0 -159677.54802816443) / Math.Sqrt((4129512.381133386 / (299)) + (5308442.070046653 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23751775554751403 = (209418.05424311716 - 159677.54802816443) / 209418.05424311716 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CreateAddAndClear<Int32>.ConcurrentBag(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.770503917378917 < 12.279213237932312.
IsChangePoint: Marked as a change because one of 4/25/2023 9:59:52 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 42.256515678485535 (T) = (0 -10044.330917948357) / Math.Sqrt((100266.36904726182 / (299)) + (48101.29734821058 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21088985266283175 = (12728.680466019467 - 10044.330917948357) / 12728.680466019467 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CreateAddAndClear<Int32>.ICollection(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.482903341070776 < 1.5121843597438678.
IsChangePoint: Marked as a change because one of 5/19/2023 12:51:11 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 15.759163734130171 (T) = (0 -1473.4616471054062) / Math.Sqrt((6222.901213661253 / (299)) + (373.81482626802114 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.0674501582465381 = (1580.035276543369 - 1473.4616471054062) / 1580.035276543369 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CreateAddAndClear<Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 30.31437460468058 < 46.75763466726021.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 126.14919379602334 (T) = (0 -29840.861949543392) / Math.Sqrt((982606.5107198489 / (299)) + (291200.0884767521 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.40388297880418883 = (50058.73157200343 - 29840.861949543392) / 50058.73157200343 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.AddGivenSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ObservableCollection - Duration of single invocation 13.94 μs 8.95 μs 0.64 0.04 True
IDictionary - Duration of single invocation 4.87 μs 3.77 μs 0.77 0.01 False
SortedList - Duration of single invocation 56.64 μs 53.48 μs 0.94 0.04 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<Int32>.ObservableCollection(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.954306707665905 < 13.239596192312398.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 15.136262076236024 (T) = (0 -9658.62213771622) / Math.Sqrt((95846.0640631844 / (299)) + (1118316.0519195234 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3152990467873014 = (14106.336631191782 - 9658.62213771622) / 14106.336631191782 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.AddGivenSize<Int32>.IDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.774369886344186 < 4.631678500592173.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 46.65796096661195 (T) = (0 -3823.460236781502) / Math.Sqrt((112126.10788566375 / (299)) + (8298.716696600826 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27978044845199035 = (5308.742630720753 - 3823.460236781502) / 5308.742630720753 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.AddGivenSize<Int32>.SortedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 53.476919795221846 < 53.565746251055174.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 3/31/2023 8:05:26 PM, 4/19/2023 8:38:27 PM, 4/25/2023 1:57:48 PM, 5/5/2023 3:18:41 PM, 5/8/2023 11:50:12 PM, 5/16/2023 5:40:18 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 38.24916201273488 (T) = (0 -53387.07774345483) / Math.Sqrt((6091240.991745319 / (299)) + (35249.19008966726 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09816646858963625 = (59198.372963537506 - 53387.07774345483) / 59198.372963537506 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 28.58 ns 21.12 ns 0.74 0.08 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt32>.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.12292434704909 < 27.230246478508217.
IsChangePoint: Marked as a change because one of 3/6/2023 11:36:59 AM, 3/19/2023 2:30:51 PM, 5/1/2023 3:42:23 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 59.80330184410327 (T) = (0 -21.021039319238927) / Math.Sqrt((1.871425715853401 / (299)) + (0.20156948810086808 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2956278992196449 = (29.843656919333228 - 21.021039319238927) / 29.843656919333228 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.ComponentModel.Tests.Perf_TypeDescriptorTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetConverter - Duration of single invocation 207.87 ns 172.58 ns 0.83 0.03 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.ComponentModel.Tests.Perf_TypeDescriptorTests*'

Payloads

Baseline
Compare

Histogram

System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.SomeEnum))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 172.5775200852917 < 196.62062634060706.
IsChangePoint: Marked as a change because one of 5/5/2023 7:39:36 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 55.387714826875474 (T) = (0 -174.50690006143702) / Math.Sqrt((11.120798905473558 / (299)) + (3.005096837422027 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1412035480266707 = (203.19937240129224 - 174.50690006143702) / 203.19937240129224 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Deep

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteDeepUtf8 - Duration of single invocation 4.77 ms 4.40 ms 0.92 0.10 False
WriteDeepUtf8 - Duration of single invocation 5.03 ms 4.30 ms 0.85 0.16 False
WriteDeepUtf16 - Duration of single invocation 5.47 ms 4.35 ms 0.80 0.21 False
WriteDeepUtf16 - Duration of single invocation 5.61 ms 4.39 ms 0.78 0.19 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Deep*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.396099740259739 < 4.483220086200517.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 12:29:35 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 32.33878639714565 (T) = (0 -4298971.633326474) / Math.Sqrt((57096859720.49285 / (299)) + (1441171257.9236 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11558190488656063 = (4860791.131568909 - 4298971.633326474) / 4860791.131568909 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: False)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.295967816091954 < 4.751154296568626.
IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/25/2023 12:29:35 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 39.21078916182733 (T) = (0 -4313166.307621627) / Math.Sqrt((80790889908.1158 / (299)) + (2125714144.1624076 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15918884365862893 = (5129768.170999949 - 4313166.307621627) / 5129768.170999949 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: False)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.346634126984127 < 4.824271643229167.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 12:29:35 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 30.765210684349668 (T) = (0 -4390499.468179949) / Math.Sqrt((81157015837.52419 / (299)) + (4232251917.359442 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14618183901239812 = (5142194.9881008705 - 4390499.468179949) / 5142194.9881008705 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.388417023809524 < 4.413616010781671.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/6/2023 4:44:37 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 26.851811711918852 (T) = (0 -4285720.789378787) / Math.Sqrt((114906397227.16011 / (299)) + (2103520056.457606 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1277167709196969 = (4913221.585032034 - 4285720.789378787) / 4913221.585032034 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf16 - Duration of single invocation 78.18 ns 71.14 ns 0.91 0.05 False
EncodeUtf16 - Duration of single invocation 64.36 ns 55.29 ns 0.86 0.11 False
EncodeUtf8 - Duration of single invocation 16.50 ns 14.24 ns 0.86 0.05 False
EncodeUtf16 - Duration of single invocation 59.48 ns 56.14 ns 0.94 0.08 False
EncodeUtf16 - Duration of single invocation 16.76 ns 14.41 ns 0.86 0.02 False
EncodeUtf16 - Duration of single invocation 35.65 ns 30.89 ns 0.87 0.04 False
EncodeUtf16 - Duration of single invocation 94.55 ns 83.71 ns 0.89 0.11 False
EncodeUtf8 - Duration of single invocation 40.54 ns 34.23 ns 0.84 0.01 True
EncodeUtf16 - Duration of single invocation 16.74 ns 14.30 ns 0.85 0.03 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,©2020,512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 71.1360322702233 < 74.18771980794361.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 11:25:43 AM, 3/25/2023 12:18:49 AM, 4/9/2023 11:04:55 PM, 4/25/2023 12:29:35 AM, 5/3/2023 7:05:50 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 41.24661616709247 (T) = (0 -70.70456255612282) / Math.Sqrt((5.45135182938049 / (299)) + (0.2505606999541805 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10150890548174807 = (78.69255798693565 - 70.70456255612282) / 78.69255798693565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,no <escaping /> required,512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 55.29050567979526 < 56.6173663411543.
IsChangePoint: Marked as a change because one of 3/3/2023 11:43:56 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 40.2335052947092 (T) = (0 -55.33698337418615) / Math.Sqrt((7.9236478922257385 / (299)) + (0.0029428736766863917 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10623547207721544 = (61.91449945188125 - 55.33698337418615) / 61.91449945188125 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,no <escaping /> required,16)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.244540063908174 < 15.657753233013237.
IsChangePoint: Marked as a change because one of 5/1/2023 4:40:42 AM, 5/16/2023 12:30:38 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 47.927210320084896 (T) = (0 -14.249680699448694) / Math.Sqrt((0.22600146299305843 / (299)) + (0.0008951840135172777 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08808170173311658 = (15.626049753065006 - 14.249680699448694) / 15.626049753065006 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,no escaping required,512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 56.143636571181986 < 56.48501753368244.
IsChangePoint: Marked as a change because one of 3/20/2023 11:25:43 AM, 3/25/2023 12:18:49 AM, 4/3/2023 9:14:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 22.14454733260377 (T) = (0 -55.3290540810164) / Math.Sqrt((13.603856614717788 / (299)) + (0.06256770827063009 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08237739530117785 = (60.29608882529245 - 55.3290540810164) / 60.29608882529245 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,no <escaping /> required,16)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.410104195594416 < 15.924445502795349.
IsChangePoint: Marked as a change because one of 5/2/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 55.123504314234374 (T) = (0 -14.36906456201862) / Math.Sqrt((0.22260402560153933 / (299)) + (0.004838229006852902 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11362817121680323 = (16.211102491540533 - 14.36906456201862) / 16.211102491540533 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,©2020,16)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 30.888341712928643 < 33.66196135831252.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 5.585413026951016 (T) = (0 -31.49971635181943) / Math.Sqrt((0.8193572278679268 / (299)) + (3.6638771070543115 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08641652600751248 = (34.47929745725507 - 31.49971635181943) / 34.47929745725507 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,&Hello+<World>!,512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 83.70755136664567 < 91.27375525036182.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 5/9/2023 7:00:05 PM, 5/19/2023 4:22:12 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 25.77011746954189 (T) = (0 -84.0406462691482) / Math.Sqrt((10.053270312522699 / (299)) + (0.4106752065429903 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.07144263951208649 = (90.50668256508006 - 84.0406462691482) / 90.50668256508006 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,©2020,16)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 34.23139343330397 < 38.48257765871931.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 47.04399236513966 (T) = (0 -34.48972694873948) / Math.Sqrt((0.26910488448274283 / (299)) + (0.1771387071173189 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14118487574736946 = (40.15966414046747 - 34.48972694873948) / 40.15966414046747 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,no escaping required,16)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.303877038110535 < 15.919216680874015.
IsChangePoint: Marked as a change because one of 5/2/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 67.41328552822424 (T) = (0 -14.331279292978751) / Math.Sqrt((0.18583471817011646 / (299)) + (0.0017614636934651616 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11459264960868747 = (16.18608574532946 - 14.331279292978751) / 16.18608574532946 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsSuffix_SecondHalf - Duration of single invocation 172.88 ns 143.90 ns 0.83 0.11 False
IsPrefix_FirstHalf - Duration of single invocation 171.49 ns 142.41 ns 0.83 0.06 False
IsPrefix_FirstHalf - Duration of single invocation 171.54 ns 142.51 ns 0.83 0.03 False
IsSuffix_SecondHalf - Duration of single invocation 173.02 ns 143.75 ns 0.83 0.05 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (, IgnoreCase, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 143.8997647309443 < 164.38960924446533.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 4/10/2023 1:33:42 PM, 4/29/2023 4:23:29 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 45.55913819930488 (T) = (0 -143.91595573188985) / Math.Sqrt((236.46949798775898 / (299)) + (0.08440728433751206 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22038188916748183 = (184.5980150181589 - 143.91595573188985) / 184.5980150181589 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (, IgnoreCase, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 142.41342639065468 < 162.8133749383947.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 5/16/2023 5:40:18 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 11.706150593625127 (T) = (0 -145.34661023893284) / Math.Sqrt((120.75253711218177 / (299)) + (66.24656300986102 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.12531117591067517 = (166.1695064987932 - 145.34661023893284) / 166.1695064987932 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreCase, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 142.5110860185013 < 162.8478822927425.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 5/5/2023 3:18:41 PM, 5/9/2023 1:22:51 PM, 5/16/2023 5:40:18 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 35.41162308897316 (T) = (0 -143.0354415235242) / Math.Sqrt((120.36968962337761 / (299)) + (0.5727428098948251 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.13903640745925266 = (166.1341347796361 - 143.0354415235242) / 166.1341347796361 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, IgnoreCase, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 143.7495184428838 < 164.37638296006574.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 4/10/2023 1:33:42 PM, 5/1/2023 10:17:01 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 45.80519388729846 (T) = (0 -143.80156460856392) / Math.Sqrt((241.15676708376597 / (299)) + (0.13636841266108 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22355402297848914 = (185.2048550244211 - 143.80156460856392) / 185.2048550244211 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToDateTime_String - Duration of single invocation 404.36 ns 355.57 ns 0.88 0.07 False
ToDateTime_String - Duration of single invocation 200.62 ns 173.19 ns 0.86 0.04 False
ToDateTime_String - Duration of single invocation 372.43 ns 334.12 ns 0.90 0.04 False
ChangeType - Duration of single invocation 31.00 ns 19.97 ns 0.64 0.08 False
ToDateTime_String - Duration of single invocation 1.01 μs 884.02 ns 0.88 0.05 False
ToDateTime_String - Duration of single invocation 259.49 ns 220.59 ns 0.85 0.05 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Perf_Convert*'

Payloads

Baseline
Compare

Histogram

System.Perf_Convert.ToDateTime_String(value: "Thursday, February 26, 2009")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 355.572474758994 < 391.39922674893245.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 19.67883248976997 (T) = (0 -351.65658078998683) / Math.Sqrt((415.0826782791323 / (299)) + (86.14624850086015 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1367603558300686 = (407.3684325841297 - 351.65658078998683) / 407.3684325841297 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Perf_Convert.ToDateTime_String(value: "12/12/1999")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 173.19355914867097 < 196.3398919103352.
IsChangePoint: Marked as a change because one of 4/17/2023 11:29:15 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 15.993824534126244 (T) = (0 -168.3492849430562) / Math.Sqrt((68.01272641770146 / (299)) + (49.955487281869445 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16083983881139494 = (200.61639330518568 - 168.3492849430562) / 200.61639330518568 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Perf_Convert.ToDateTime_String(value: "12/12/1999 11:59:59 PM")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 334.1202787044012 < 362.1225321888454.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 4.404850457326014 (T) = (0 -345.6685597645031) / Math.Sqrt((253.9155477116426 / (299)) + (609.0199872634967 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08088812621084304 = (376.0897553629026 - 345.6685597645031) / 376.0897553629026 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Perf_Convert.ChangeType

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.972877529744924 < 35.77787141884598.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 24.660227567178467 (T) = (0 -20.121798259459595) / Math.Sqrt((37.929577480823454 / (299)) + (0.08153757793528665 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.30899152399608365 = (29.119466632049754 - 20.121798259459595) / 29.119466632049754 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Perf_Convert.ToDateTime_String(value: "Fri, 27 Feb 2009 03:11:21 GMT")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 884.0214268044375 < 0.9701585995732613.
IsChangePoint: Marked as a change because one of 4/25/2023 1:57:48 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 30.579627653597086 (T) = (0 -884.2003487327777) / Math.Sqrt((668.1128602824718 / (299)) + (167.04065336933292 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11841354969814363 = (1002.9649938812312 - 884.2003487327777) / 1002.9649938812312 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Perf_Convert.ToDateTime_String(value: "February 26, 2009")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 220.58724208705132 < 250.85532974970778.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 23.07298271228282 (T) = (0 -215.8982213035664) / Math.Sqrt((99.8626837366589 / (299)) + (34.16023699219138 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1553471893043425 = (255.60587565647506 - 215.8982213035664) / 255.60587565647506 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SystemTextJson_SourceGen_ - Duration of single invocation 237.26 ns 199.21 ns 0.84 0.09 False
JsonNet_ - Duration of single invocation 626.96 ns 412.27 ns 0.66 0.09 True
DataContractJsonSerializer_ - Duration of single invocation 1.10 μs 703.06 ns 0.64 0.18 True
SystemTextJson_Reflection_ - Duration of single invocation 383.44 ns 263.34 ns 0.69 0.09 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;LoginViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>.SystemTextJson_SourceGen_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 199.20969569439947 < 221.26856254189465.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 18.096807962829832 (T) = (0 -198.6721057423533) / Math.Sqrt((89.19573896443639 / (299)) + (63.336185029943884 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17158262859858534 = (239.82126956882163 - 198.6721057423533) / 239.82126956882163 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>.JsonNet_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 412.27440083101243 < 601.3379393398344.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 101.60203043289835 (T) = (0 -408.99573378829876) / Math.Sqrt((366.5536465863643 / (299)) + (45.02286244258557 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3497798336583241 = (629.0111487766143 - 408.99573378829876) / 629.0111487766143 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>.DataContractJsonSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 703.0575151418329 < 1.022831005047816.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 64.41133292515669 (T) = (0 -683.4965808912208) / Math.Sqrt((816.2934314805992 / (299)) + (448.81116330524134 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.36515656321982004 = (1076.6380201673053 - 683.4965808912208) / 1076.6380201673053 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>.SystemTextJson_Reflection_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 263.3407149326892 < 330.48643374128017.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 16.089736363290733 (T) = (0 -269.7914628054328) / Math.Sqrt((103.00836914168507 / (299)) + (272.1345767374125 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2157458015239963 = (344.0102244013524 - 269.7914628054328) / 344.0102244013524 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Decimal

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 59.01 ns 50.53 ns 0.86 0.06 False
TryParse - Duration of single invocation 76.28 ns 54.54 ns 0.71 0.08 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Decimal.ToString(value: 123456.789)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 50.533679538564314 < 60.56345800347168.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/17/2023 9:25:32 PM, 5/2/2023 3:18:41 PM, 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 55.31969399594897 (T) = (0 -50.14236927437589) / Math.Sqrt((16.3287330834763 / (299)) + (0.24355798005267357 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23005185006613882 = (65.12434542336797 - 50.14236927437589) / 65.12434542336797 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Decimal.TryParse(value: "123456.789")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 54.53809108165411 < 72.48894039644783.
IsChangePoint: Marked as a change because one of 3/15/2023 5:33:41 AM, 3/20/2023 8:31:51 AM, 5/11/2023 5:02:01 PM, 5/19/2023 3:32:16 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 9.63477999737064 (T) = (0 -59.62851244309874) / Math.Sqrt((28.01209228409507 / (299)) + (54.20192631321986 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.2575799921960219 = (80.31641364229306 - 59.62851244309874) / 80.31641364229306 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SystemTextJson_Reflection_ - Duration of single invocation 140.73 μs 105.64 μs 0.75 0.19 False
SystemTextJson_SourceGen_ - Duration of single invocation 133.85 μs 104.34 μs 0.78 0.25 False
JsonNet_ - Duration of single invocation 412.90 μs 322.47 μs 0.78 0.11 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;CollectionsOfPrimitives&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_Reflection_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 105.63511073494146 < 132.33835481185528.
IsChangePoint: Marked as a change because one of 4/7/2023 11:52:19 PM, 4/24/2023 6:11:43 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 14.981701597986856 (T) = (0 -109991.57976077999) / Math.Sqrt((71537072.25936686 / (299)) + (18026195.62316245 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1479780729457658 = (129094.77593031316 - 109991.57976077999) / 129094.77593031316 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_SourceGen_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 104.3396552743614 < 127.44049334490738.
IsChangePoint: Marked as a change because one of 4/5/2023 12:23:38 PM, 5/2/2023 4:25:00 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 18.698807473091943 (T) = (0 -105769.25651075978) / Math.Sqrt((47835269.967427745 / (299)) + (13711852.91321704 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16307363403507127 = (126378.21057150447 - 105769.25651075978) / 126378.21057150447 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.JsonNet_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 322.4749765857812 < 401.95726850865805.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 38.11652068987393 (T) = (0 -307085.35830662906) / Math.Sqrt((146193893.3392018 / (299)) + (94665605.54252918 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2570641279982546 = (413340.3297369758 - 307085.35830662906) / 413340.3297369758 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.AddGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentDictionary - Duration of single invocation 33.03 μs 28.06 μs 0.85 0.02 True
ObservableCollection - Duration of single invocation 15.29 μs 12.48 μs 0.82 0.03 False
List - Duration of single invocation 1.87 μs 1.72 μs 0.92 0.04 False
ICollection - Duration of single invocation 2.59 μs 2.30 μs 0.89 0.20 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<String>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 28.055844141323792 < 31.230896873792698.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 89.21998515295704 (T) = (0 -27802.586937293792) / Math.Sqrt((620428.412261817 / (299)) + (20110.888489735375 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16186886024575825 = (33172.12023102508 - 27802.586937293792) / 33172.12023102508 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.AddGivenSize<String>.ObservableCollection(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.48268189809389 < 14.674858623640281.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 30.32464770362576 (T) = (0 -12528.123667715361) / Math.Sqrt((1575991.0067016978 / (299)) + (78455.8977241505 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2046910967709978 = (15752.525360712576 - 12528.123667715361) / 15752.525360712576 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.AddGivenSize<String>.List(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.7243897683807905 < 1.774934866940063.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 89.66352048278577 (T) = (0 -1723.87026846464) / Math.Sqrt((853.513247344804 / (299)) + (1.4531704961787213 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08221703110484949 = (1878.2983852270402 - 1723.87026846464) / 1878.2983852270402 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.AddGivenSize<String>.ICollection(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.303040043511355 < 2.5219673676961483.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/19/2023 12:51:11 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 10.960261394406217 (T) = (0 -2362.7393562231186) / Math.Sqrt((6661.06640760926 / (299)) + (6443.160594427195 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.08975281102515385 = (2595.7117855910356 - 2362.7393562231186) / 2595.7117855910356 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IndexerSet<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 18.23 μs 10.51 μs 0.58 0.04 False
SortedDictionary - Duration of single invocation 35.61 μs 15.57 μs 0.44 0.02 True
IList - Duration of single invocation 2.08 μs 994.41 ns 0.48 0.18 True
ConcurrentDictionary - Duration of single invocation 12.93 μs 10.94 μs 0.85 0.01 True

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<Int32>.SortedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.51009316415181 < 17.014216217299328.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 49.20461762350057 (T) = (0 -10815.951205761663) / Math.Sqrt((1285205.1853174109 / (299)) + (313464.1234209975 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.43400615059092373 = (19109.66208741317 - 10815.951205761663) / 19109.66208741317 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.IndexerSet<Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 15.567526554034789 < 33.524341808007804.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 240.5285520998286 (T) = (0 -15890.1092563979) / Math.Sqrt((768525.1463547825 / (299)) + (53090.84059206836 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5525281137239828 = (35510.854969324915 - 15890.1092563979) / 35510.854969324915 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.IndexerSet<Int32>.IList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 994.4052535831058 < 1.973266042810313.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 222.55659630340259 (T) = (0 -994.7979405550407) / Math.Sqrt((6761.949871153559 / (299)) + (0.8894480076063246 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5158605983404554 = (2054.775829327356 - 994.7979405550407) / 2054.775829327356 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.IndexerSet<Int32>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.94333521303258 < 12.245409405201885.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 195.33484225755447 (T) = (0 -10948.96207287745) / Math.Sqrt((19317.49467397934 / (299)) + (525.9832191575514 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15459795599324533 = (12951.189496757319 - 10948.96207287745) / 12951.189496757319 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Formats.Cbor.Tests.Perf_CborWriter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteCoseKey - Duration of single invocation 645.31 ns 499.55 ns 0.77 0.04 False
WriteCoseKey - Duration of single invocation 671.15 ns 530.29 ns 0.79 0.03 False
WriteCoseKey - Duration of single invocation 716.64 ns 570.84 ns 0.80 0.06 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Formats.Cbor.Tests.Perf_CborWriter*'

Payloads

Baseline
Compare

Histogram

System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey: ECDSA_P256)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 499.55057574464973 < 602.6855318914054.
IsChangePoint: Marked as a change because one of 3/17/2023 10:06:19 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 82.6306085809626 (T) = (0 -500.7252699256509) / Math.Sqrt((546.0453467972378 / (299)) + (12.873674770191897 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2168826485405148 = (639.4000452070893 - 500.7252699256509) / 639.4000452070893 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey: ECDSA_P384)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 530.2885948820756 < 635.6882105204986.
IsChangePoint: Marked as a change because one of 3/17/2023 10:06:19 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 52.48332767727143 (T) = (0 -536.7991912233107) / Math.Sqrt((801.0934437890576 / (299)) + (55.889656737003406 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20526362076230104 = (675.4430843321928 - 536.7991912233107) / 675.4430843321928 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey: ECDSA_P521)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 570.8389333095333 < 679.8023848592392.
IsChangePoint: Marked as a change because one of 3/17/2023 10:06:19 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 64.74825031002923 (T) = (0 -577.3161963156108) / Math.Sqrt((540.877473306483 / (299)) + (36.47817377334651 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1941555709224086 = (716.4114753221473 - 577.3161963156108) / 716.4114753221473 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<XmlElement>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractSerializer_ - Duration of single invocation 368.07 ns 238.75 ns 0.65 0.04 True
DataContractSerializer_BinaryXml_ - Duration of single invocation 214.37 ns 121.53 ns 0.57 0.20 True
XmlSerializer_ - Duration of single invocation 857.13 ns 696.27 ns 0.81 0.04 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;XmlElement&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<XmlElement>.DataContractSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 238.7540646103402 < 344.03944750283927.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 87.88631606018112 (T) = (0 -236.80163686197045) / Math.Sqrt((77.82756899285603 / (299)) + (22.2775235259231 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.34272800904882866 = (360.2795191672216 - 236.80163686197045) / 360.2795191672216 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_ToStream<XmlElement>.DataContractSerializer_BinaryXml_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 121.53227930339277 < 203.97965344669933.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 31.60095496561112 (T) = (0 -128.58302823531085) / Math.Sqrt((62.892824292615686 / (299)) + (98.17837395743676 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.406432383681896 = (216.62743165287642 - 128.58302823531085) / 216.62743165287642 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_ToStream<XmlElement>.XmlSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 696.2740384615383 < 808.8174187444426.
IsChangePoint: Marked as a change because one of 4/26/2023 1:55:38 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 49.157648118590046 (T) = (0 -701.3707354732883) / Math.Sqrt((450.64593457940083 / (299)) + (129.51944188469795 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1918354823295415 = (867.8563833698066 - 701.3707354732883) / 867.8563833698066 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Guids

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteGuids - Duration of single invocation 2.14 ms 1.18 ms 0.55 0.02 True
WriteGuids - Duration of single invocation 2.60 ms 1.87 ms 0.72 0.01 True
WriteGuids - Duration of single invocation 2.17 ms 1.27 ms 0.58 0.02 True
WriteGuids - Duration of single invocation 2.55 ms 1.91 ms 0.75 0.03 True

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Guids*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: False, SkipValidation: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.1828320535714287 < 2.0223120882626486.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 43.02333587552965 (T) = (0 -1261530.8871997397) / Math.Sqrt((273968512.7490956 / (299)) + (5115805355.817319 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4038123058917703 = (2115996.186547799 - 1261530.8871997397) / 2115996.186547799 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: True, SkipValidation: False)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.869349094202899 < 2.4551429140370296.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 44.37503536364519 (T) = (0 -1841160.9509450223) / Math.Sqrt((1017173116.4714483 / (299)) + (3472887619.0022635 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.283889090251847 = (2571055.5807514996 - 1841160.9509450223) / 2571055.5807514996 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: False, SkipValidation: False)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2662468489583334 < 2.047984084334936.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 59.018328485783826 (T) = (0 -1272854.844460603) / Math.Sqrt((480200920.17746097 / (299)) + (2810310039.1424537 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.40626720582317305 = (2143817.651550368 - 1272854.844460603) / 2143817.651550368 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: True, SkipValidation: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.907696796875 < 2.4235971671981296.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 34.86625236932225 (T) = (0 -1838182.0179598124) / Math.Sqrt((3085763988.027898 / (299)) + (5215911036.645751 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.277869412703188 = (2545498.0723649613 - 1838182.0179598124) / 2545498.0723649613 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeObjectProperty - Duration of single invocation 5.87 μs 4.29 μs 0.73 0.29 False
SerializeToString - Duration of single invocation 5.84 μs 4.52 μs 0.77 0.10 False
SerializeToUtf8Bytes - Duration of single invocation 5.36 μs 3.93 μs 0.73 0.17 False
SerializeToUtf8Bytes - Duration of single invocation 3.83 μs 3.15 μs 0.82 0.08 False
SerializeObjectProperty - Duration of single invocation 4.40 μs 4.10 μs 0.93 0.15 False
SerializeToWriter - Duration of single invocation 3.59 μs 3.18 μs 0.89 0.08 False
SerializeToWriter - Duration of single invocation 5.44 μs 3.67 μs 0.67 0.31 False
SerializeToStream - Duration of single invocation 5.61 μs 3.86 μs 0.69 0.18 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeObjectProperty(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.290384846766015 < 5.561032059197228.
IsChangePoint: Marked as a change because one of 3/20/2023 4:51:16 PM, 4/19/2023 3:30:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 33.688506619659805 (T) = (0 -4482.612067472962) / Math.Sqrt((289036.7908823775 / (299)) + (37181.487448457 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.31736379831505 = (6566.619315542505 - 4482.612067472962) / 6566.619315542505 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.515272086914996 < 5.408530837357954.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 50.958026084280014 (T) = (0 -4441.139338676139) / Math.Sqrt((56510.269420062614 / (299)) + (6952.837787174644 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2358830316304873 = (5812.119770292141 - 4441.139338676139) / 5812.119770292141 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.931415411961522 < 5.037549806734175.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 89.64901827369465 (T) = (0 -3837.445702137651) / Math.Sqrt((62720.25338039582 / (299)) + (2219.7887203333507 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.31305163115470697 = (5586.221434062213 - 3837.445702137651) / 5586.221434062213 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.1541870873753703 < 3.5770574522894667.
IsChangePoint: Marked as a change because one of 4/4/2023 8:16:00 PM, 5/7/2023 10:29:39 PM, 5/8/2023 11:50:12 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 85.59091890280793 (T) = (0 -3161.638450077731) / Math.Sqrt((28253.49925865225 / (299)) + (167.00490170582847 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21904051780689596 = (4048.4026664215185 - 3161.638450077731) / 4048.4026664215185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeObjectProperty(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.097175088803708 < 4.178993851076222.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 5/6/2023 11:12:53 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 19.22630511704932 (T) = (0 -3840.059197083945) / Math.Sqrt((34159.774845613036 / (299)) + (23805.731925669523 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1808887668332779 = (4688.080252834646 - 3840.059197083945) / 4688.080252834646 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToWriter(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.1841383237280243 < 3.4227300524120365.
IsChangePoint: Marked as a change because one of 5/12/2023 4:02:48 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 24.745056836306794 (T) = (0 -3205.009330331277) / Math.Sqrt((11161.647114962288 / (299)) + (7550.824462013996 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16104560118000077 = (3820.242595830198 - 3205.009330331277) / 3820.242595830198 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToWriter(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.667549589201878 < 4.903694784906965.
IsChangePoint: Marked as a change because one of 3/15/2023 4:14:49 PM, 4/12/2023 9:47:42 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 87.04208944418905 (T) = (0 -3663.35479380461) / Math.Sqrt((129147.77731461175 / (299)) + (1080.508372624325 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3503246693299906 = (5638.746956924771 - 3663.35479380461) / 5638.746956924771 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.8604351486775093 < 4.925697449352776.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 59.662525249368564 (T) = (0 -3922.0355310397495) / Math.Sqrt((69340.95612692933 / (299)) + (6191.401550411322 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2881637472716548 = (5509.744012063541 - 3922.0355310397495) / 5509.744012063541 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 6.80 secs 4.07 secs 0.60 0.05 True
Count - Duration of single invocation 19.37 ms 11.19 ms 0.58 0.30 False
Count - Duration of single invocation 470.71 ms 330.76 ms 0.70 0.24 False
Count - Duration of single invocation 61.44 ms 52.98 ms 0.86 0.02 True
Count - Duration of single invocation 5.70 ms 4.84 ms 0.85 0.12 False
Count - Duration of single invocation 1.17 secs 783.25 ms 0.67 0.03 False
Count - Duration of single invocation 61.47 ms 53.01 ms 0.86 0.02 True
Count - Duration of single invocation 348.02 ms 260.98 ms 0.75 0.20 True
Count - Duration of single invocation 30.25 ms 24.83 ms 0.82 0.01 False
Count - Duration of single invocation 6.64 secs 3.82 secs 0.58 0.01 True
Count - Duration of single invocation 348.30 ms 258.98 ms 0.74 0.15 True

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{0,2}(Tom|Sawyer|Huckleberry|Finn)", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.068148184210527 < 6.435229490999999.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 112.76015001908563 (T) = (0 -3943419445.7815137) / Math.Sqrt((38653194976652056 / (299)) + (7144273785667144 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.42693878819463943 = (6881323259.269711 - 3943419445.7815137) / 6881323259.269711 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom|Sawyer|Huckleberry|Finn", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.188343030303031 < 18.133261958333332.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 133.87993092161773 (T) = (0 -11235216.193486003) / Math.Sqrt((640197212108.9498 / (299)) + (6097084678.338539 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.37841373068002776 = (18075071.390778232 - 11235216.193486003) / 18075071.390778232 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Tom|Sawyer|Huckleberry|Finn", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 330.76050714285714 < 442.9350820238095.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 104.27195027337186 (T) = (0 -332895279.31530005) / Math.Sqrt((535379751945863.75 / (299)) + (5700414326557.698 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.31863859591634436 = (488573725.0747301 - 332895279.31530005) / 488573725.0747301 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{0,2}(Tom|Sawyer|Huckleberry|Finn)", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 52.976888461538465 < 58.41778033928571.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 55.26153133407696 (T) = (0 -53428392.928994074) / Math.Sqrt((716033881044.0577 / (299)) + (251511091409.322 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13232854955400886 = (61576755.69656163 - 53428392.928994074) / 61576755.69656163 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Huck[a-zA-Z]+|Saw[a-zA-Z]+", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.841101666666667 < 5.4414157395833325.
IsChangePoint: Marked as a change because one of 3/6/2023 6:03:21 AM, 3/28/2023 3:45:48 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 18.352520666778045 (T) = (0 -4830314.331122271) / Math.Sqrt((38351887779.534386 / (299)) + (31311493232.675602 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16062812411612695 = (5754677.360420096 - 4830314.331122271) / 5754677.360420096 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "([A-Za-z]awyer|[A-Za-z]inn)\s", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 783.24715 < 1.1106218959999998.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/28/2023 3:45:48 AM, 5/9/2023 1:22:51 PM, 5/16/2023 5:40:18 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 50.19957319881781 (T) = (0 -785295598.6644124) / Math.Sqrt((6804439892166320 / (299)) + (69221051355520.31 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25303530096031196 = (1051315543.6582254 - 785295598.6644124) / 1051315543.6582254 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{2,4}(Tom|Sawyer|Huckleberry|Finn)", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 53.011803846153846 < 58.44326296703296.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 80.02314275141978 (T) = (0 -53348639.47978304) / Math.Sqrt((362469059398.04694 / (299)) + (117030596422.81075 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13164405302205595 = (61436372.56754813 - 53348639.47978304) / 61436372.56754813 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{2,4}(Tom|Sawyer|Huckleberry|Finn)", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 260.97727857142854 < 330.7346878333333.
IsChangePoint: Marked as a change because one of 5/15/2023 6:46:12 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 40.4615826383994 (T) = (0 -263825187.1042648) / Math.Sqrt((302729365554047.6 / (299)) + (110944361314769.86 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.25488159104771113 = (354071492.44269705 - 263825187.1042648) / 354071492.44269705 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Tom|Sawyer|Huckleberry|Finn", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 24.829897692307693 < 28.923753041666668.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/15/2023 9:18:21 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 38.99055318971435 (T) = (0 -24827461.832051277) / Math.Sqrt((2715542513987.3457 / (299)) + (2581163370.6910367 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.13071604528547598 = (28560819.163175173 - 24827461.832051277) / 28560819.163175173 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{2,4}(Tom|Sawyer|Huckleberry|Finn)", Options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.8179236933333334 < 6.5107857009880945.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 110.35900490741157 (T) = (0 -3908069591.902193) / Math.Sqrt((14331382405026046 / (299)) + (7917329005066192 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4198835868949723 = (6736698882.530416 - 3908069591.902193) / 6736698882.530416 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{0,2}(Tom|Sawyer|Huckleberry|Finn)", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 258.97780666666665 < 330.8630889285714.
IsChangePoint: Marked as a change because one of 5/15/2023 6:46:12 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 32.65935188745923 (T) = (0 -259485679.51403904) / Math.Sqrt((130273230765704.19 / (299)) + (217701732618383.06 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.2618362035555529 = (351528591.3017105 - 259485679.51403904) / 351528591.3017105 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetDateTimeOffset - Duration of single invocation 5.37 μs 4.76 μs 0.89 0.01 True
GetSingle - Duration of single invocation 4.93 μs 4.58 μs 0.93 0.00 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Get.GetDateTimeOffset

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.759445668576678 < 5.181068151396452.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 22.86433989950129 (T) = (0 -4881.639539957722) / Math.Sqrt((3638.2996765467738 / (299)) + (7271.767635845437 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10069780759049769 = (5428.25268431553 - 4881.639539957722) / 5428.25268431553 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetSingle

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.575950612745099 < 4.636296040085683.
IsChangePoint: Marked as a change because one of 5/15/2023 2:12:16 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 47.68449766902568 (T) = (0 -4553.238206597347) / Math.Sqrt((2519.678520602618 / (299)) + (575.62672072175 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0706581634945991 = (4899.422395228503 - 4553.238206597347) / 4899.422395228503 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_DateTime

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 186.56 ns 150.64 ns 0.81 0.18 False
ToString - Duration of single invocation 189.57 ns 147.24 ns 0.78 0.15 False
ToString - Duration of single invocation 44.20 ns 35.12 ns 0.79 0.02 False
GetNow - Duration of single invocation 104.47 ns 85.64 ns 0.82 0.04 False
ParseO - Duration of single invocation 38.64 ns 35.12 ns 0.91 0.03 False
ToString - Duration of single invocation 31.97 ns 28.11 ns 0.88 0.04 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTime*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_DateTime.ToString(format: null)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 150.64081353973822 < 179.78673340937644.
IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 37.94558887676164 (T) = (0 -147.1805517974115) / Math.Sqrt((179.1850950304548 / (299)) + (23.62509740670073 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2861152660359295 = (206.1685098379188 - 147.1805517974115) / 206.1685098379188 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_DateTime.ToString(format: "G")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 147.23540195917474 < 188.70816010346886.
IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 16.12637127104522 (T) = (0 -144.928391964673) / Math.Sqrt((106.466175617421 / (299)) + (191.7940647355151 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3019302007297932 = (207.613038289563 - 144.928391964673) / 207.613038289563 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_DateTime.ToString(format: "o")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 35.12173592861791 < 41.93749153292947.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/19/2023 8:38:27 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 7.829337878958852 (T) = (0 -37.338100767840565) / Math.Sqrt((1.9375564046647842 / (299)) + (5.872792649502249 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12430010151533506 = (42.63800970224096 - 37.338100767840565) / 42.63800970224096 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_DateTime.GetNow

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 85.63915389804374 < 99.32710028899965.
IsChangePoint: Marked as a change because one of 3/11/2023 9:11:51 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 52.63160851818624 (T) = (0 -86.53909777582076) / Math.Sqrt((19.427371950369682 / (299)) + (0.38538921040586854 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1575961682571567 = (102.72875610831522 - 86.53909777582076) / 102.72875610831522 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_DateTime.ParseO

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 35.11691774543474 < 38.013550394670666.
IsChangePoint: Marked as a change because one of 3/3/2023 11:00:07 PM, 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 33.065145524564215 (T) = (0 -34.96403165062574) / Math.Sqrt((3.2166112388401977 / (299)) + (0.11726007172886262 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11738446341520052 = (39.614113055290055 - 34.96403165062574) / 39.614113055290055 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_DateTime.ToString(format: "r")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 28.110206307472556 < 30.349216851036157.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/9/2023 4:09:45 AM, 4/19/2023 8:38:27 PM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.68601908406246 (T) = (0 -27.839273901203963) / Math.Sqrt((1.5267042506547694 / (299)) + (0.514848100311496 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.06528595699577473 = (29.783733441863053 - 27.839273901203963) / 29.783733441863053 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.Perf_FileStream

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetLength - Duration of single invocation 29.10 ns 22.81 ns 0.78 0.01 False
SeekBackward - Duration of single invocation 36.03 μs 28.36 μs 0.79 0.01 True
SeekBackward - Duration of single invocation 32.54 μs 24.82 μs 0.76 0.01 False
ReadAsync - Duration of single invocation 99.89 ms 84.77 ms 0.85 0.25 False
SeekForward - Duration of single invocation 29.47 μs 22.61 μs 0.77 0.02 False
SeekForward - Duration of single invocation 31.52 μs 24.11 μs 0.76 0.00 True

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_FileStream*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_FileStream.GetLength(options: Read)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 22.809519708364476 < 27.6020905293708.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 19.743205197618014 (T) = (0 -22.925241024575733) / Math.Sqrt((5.539584759918167 / (299)) + (0.02228266531553846 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10915034629792277 = (25.734130253411443 - 22.925241024575733) / 25.734130253411443 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_FileStream.SeekBackward(fileSize: 1024, options: Asynchronous)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 28.360956060606064 < 34.15540535499086.
IsChangePoint: Marked as a change because one of 4/8/2023 6:29:53 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 145.20927652970337 (T) = (0 -28281.463394400682) / Math.Sqrt((372975.05387607624 / (299)) + (25744.03703852285 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22582799912159646 = (36531.2402958406 - 28281.463394400682) / 36531.2402958406 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_FileStream.SeekBackward(fileSize: 1024, options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 24.821682353275705 < 30.876383922686195.
IsChangePoint: Marked as a change because one of 4/4/2023 8:16:00 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 91.10584325171399 (T) = (0 -24726.187281142946) / Math.Sqrt((988220.6562252571 / (274)) + (25005.2387381021 / (13))) is greater than 1.968322603028609 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (274) + (13) - 2, .975) and 0.21507296605573903 = (31501.255800674582 - 24726.187281142946) / 31501.255800674582 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_FileStream.ReadAsync(fileSize: 104857600, userBufferSize: 4096, options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 84.7713 < 91.40398483516483.
IsChangePoint: Marked as a change because one of 5/13/2023 2:42:33 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.070630945864556 (T) = (0 -88790425.72061902) / Math.Sqrt((13717677737479.771 / (252)) + (7524802171586.796 / (35))) is greater than 1.968322603028609 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (252) + (35) - 2, .975) and 0.06591338519651178 = (95055880.59336299 - 88790425.72061902) / 95055880.59336299 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_FileStream.SeekForward(fileSize: 1024, options: None)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 22.61326816725243 < 27.926014596325746.
IsChangePoint: Marked as a change because one of 4/4/2023 8:16:00 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 76.00600561882243 (T) = (0 -22433.057788300015) / Math.Sqrt((1109220.9119964105 / (273)) + (31139.33923502776 / (13))) is greater than 1.968352158354924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (273) + (13) - 2, .975) and 0.21401206445601587 = (28541.22407460878 - 22433.057788300015) / 28541.22407460878 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_FileStream.SeekForward(fileSize: 1024, options: Asynchronous)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 24.108067948717952 < 30.004070216263067.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 126.07689392044011 (T) = (0 -23979.02667528022) / Math.Sqrt((311079.4812564483 / (299)) + (36219.69061039767 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2454207802377828 = (31778.01090631211 - 23979.02667528022) / 31778.01090631211 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.Puzzle

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 418.66 ms 291.62 ms 0.70 0.01 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Puzzle*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.Puzzle.Test

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 291.6168666666667 < 397.6498149880952.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 51.66160347526166 (T) = (0 -300620773.796492) / Math.Sqrt((38605308826063.14 / (299)) + (75428464026851.95 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2950436262712472 = (426438833.66343224 - 300620773.796492) / 426438833.66343224 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Ctor

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Ctor - Duration of single invocation 13.23 ns 11.82 ns 0.89 0.09 False
Ctor - Duration of single invocation 13.26 ns 11.82 ns 0.89 0.08 False
Ctor - Duration of single invocation 13.14 ns 11.88 ns 0.90 0.06 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Ctor*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Ctor.Ctor(Formatted: True, SkipValidation: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.817417282481314 < 12.34544655131969.
IsChangePoint: Marked as a change because one of 5/10/2023 2:00:36 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 54.22149305710874 (T) = (0 -11.786675678762565) / Math.Sqrt((0.1387359227504965 / (299)) + (0.005879817824233286 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1222300851060788 = (13.427978652226864 - 11.786675678762565) / 13.427978652226864 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Ctor.Ctor(Formatted: False, SkipValidation: False)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.824516398819359 < 12.11401154704218.
IsChangePoint: Marked as a change because one of 5/10/2023 6:05:58 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 61.348918187035615 (T) = (0 -11.765629830850516) / Math.Sqrt((0.11019696847705286 / (299)) + (0.004553199737967977 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12265011057844231 = (13.410419232636674 - 11.765629830850516) / 13.410419232636674 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Ctor.Ctor(Formatted: False, SkipValidation: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.884279228951756 < 12.110846593190754.
IsChangePoint: Marked as a change because one of 5/11/2023 5:02:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 70.21336020588997 (T) = (0 -11.787939910596712) / Math.Sqrt((0.1024526820089847 / (299)) + (0.0024543311600376968 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12073449633889148 = (13.40657612690795 - 11.787939910596712) / 13.40657612690795 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Binary_FromStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinaryFormatter_ - Duration of single invocation 2.58 μs 2.09 μs 0.81 0.01 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_FromStream&lt;LoginViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_FromStream<LoginViewModel>.BinaryFormatter_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.085600440379404 < 2.450235588501411.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 79.570187910771 (T) = (0 -2106.144544573021) / Math.Sqrt((910.929973710803 / (299)) + (448.2601783851201 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1879433844180976 = (2593.5932349541945 - 2106.144544573021) / 2593.5932349541945 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchF.Regula

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 220.47 ms 191.98 ms 0.87 0.00 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.Regula*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchF.Regula.Test

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 191.98305333333334 < 209.46903083333336.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 1252.256437738052 (T) = (0 -191995417.35136658) / Math.Sqrt((50340842417.28541 / (299)) + (4470794603.979201 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12863329223487519 = (220338252.1278499 - 191995417.35136658) / 220338252.1278499 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromUtf8Bytes - Duration of single invocation 38.25 μs 26.11 μs 0.68 0.06 True
DeserializeFromReader - Duration of single invocation 44.07 μs 29.52 μs 0.67 0.05 True
DeserializeFromStream - Duration of single invocation 35.84 μs 24.79 μs 0.69 0.06 True
DeserializeFromUtf8Bytes - Duration of single invocation 35.77 μs 24.20 μs 0.68 0.05 True
DeserializeFromReader - Duration of single invocation 46.61 μs 32.78 μs 0.70 0.05 True
DeserializeFromString - Duration of single invocation 38.66 μs 26.23 μs 0.68 0.04 True
DeserializeFromStream - Duration of single invocation 39.44 μs 27.31 μs 0.69 0.05 True
DeserializeFromString - Duration of single invocation 34.86 μs 24.87 μs 0.71 0.05 True

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;TreeRecord&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 26.110673837488015 < 36.579823156927866.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 125.02000314895744 (T) = (0 -26442.796158947123) / Math.Sqrt((712971.4617252888 / (299)) + (89449.6875755263 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.31275914820149614 = (38476.75249477172 - 26442.796158947123) / 38476.75249477172 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromReader(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 29.518344845013477 < 41.59485165757559.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 110.19966587730323 (T) = (0 -29579.16432106788) / Math.Sqrt((773088.2474784927 / (299)) + (157773.6622319574 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.31131430025820295 = (42950.164831588256 - 29579.16432106788) / 42950.164831588256 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 24.788293212312553 < 34.253290465431185.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 29.166237410886808 (T) = (0 -25116.20127389101) / Math.Sqrt((617664.4252621657 / (299)) + (1565782.2927148433 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2889928891997511 = (35324.82431240717 - 25116.20127389101) / 35324.82431240717 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 24.196855247465717 < 33.535298545725105.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 125.28798376325602 (T) = (0 -24338.520046134123) / Math.Sqrt((516837.61322131276 / (299)) + (67682.6913231278 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.30005445108248036 = (34772.019171740074 - 24338.520046134123) / 34772.019171740074 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromReader(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 32.78448680992312 < 44.0940273179399.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 64.98372239693968 (T) = (0 -32521.81678762803) / Math.Sqrt((745691.9552773691 / (299)) + (602988.0975243613 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.306402584131227 = (46888.607200031845 - 32521.81678762803) / 46888.607200031845 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 26.23042830177373 < 36.71028602096864.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 134.84092315775263 (T) = (0 -26731.639235902676) / Math.Sqrt((555166.3014618539 / (299)) + (77925.42355035612 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.30889104789619165 = (38679.34159227536 - 26731.639235902676) / 38679.34159227536 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 27.3054726692765 < 37.14623712036137.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 133.21718509277608 (T) = (0 -27148.97604579065) / Math.Sqrt((647206.5119008519 / (299)) + (75443.24131734837 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3045919176570065 = (39040.35160810809 - 27148.97604579065) / 39040.35160810809 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 24.868826963906585 < 33.87633866209979.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 106.22879538680021 (T) = (0 -24502.512288979466) / Math.Sqrt((535759.941409763 / (299)) + (105066.58936800442 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3010902846278854 = (35058.193855459285 - 24502.512288979466) / 35058.193855459285 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractSerializer_ - Duration of single invocation 571.81 μs 410.13 μs 0.72 0.11 True
XmlSerializer_ - Duration of single invocation 409.93 μs 365.42 μs 0.89 0.15 False
DataContractSerializer_BinaryXml_ - Duration of single invocation 325.74 μs 227.69 μs 0.70 0.23 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;CollectionsOfPrimitives&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>.DataContractSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 410.1348026315789 < 576.2402174496225.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 64.29750834361018 (T) = (0 -405530.9389633703) / Math.Sqrt((337300143.9937539 / (299)) + (102303199.26888293 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3223050150158017 = (598397.4323977416 - 405530.9389633703) / 598397.4323977416 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>.XmlSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 365.4220448617631 < 389.6408428870808.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 11.717365083046026 (T) = (0 -367992.39660760236) / Math.Sqrt((268250230.43231246 / (299)) + (111849484.01623325 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08937471448377836 = (404109.5744436662 - 367992.39660760236) / 404109.5744436662 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>.DataContractSerializer_BinaryXml_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 227.68769277474195 < 310.0126742792462.
IsChangePoint: Marked as a change because one of 4/1/2023 5:53:03 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 63.98942390460196 (T) = (0 -223079.62034236174) / Math.Sqrt((215874029.17358837 / (299)) + (31166019.345250368 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.33626242976594095 = (336096.11742137093 - 223079.62034236174) / 336096.11742137093 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
JsonNet_ - Duration of single invocation 1.46 μs 875.00 ns 0.60 0.03 True
SystemTextJson_SourceGen_ - Duration of single invocation 557.93 ns 398.78 ns 0.71 0.08 True
DataContractJsonSerializer_ - Duration of single invocation 4.60 μs 3.47 μs 0.75 0.01 True
SystemTextJson_Reflection_ - Duration of single invocation 569.57 ns 400.42 ns 0.70 0.10 True

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;LoginViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>.JsonNet_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 874.9972721173243 < 1.385107905388047.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 190.4037407972846 (T) = (0 -865.8324488469859) / Math.Sqrt((1308.294415652561 / (299)) + (83.30613655347251 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4193297733232955 = (1491.0915164401035 - 865.8324488469859) / 1491.0915164401035 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>.SystemTextJson_SourceGen_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 398.77758452629917 < 509.67641833714356.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 71.2797154200804 (T) = (0 -397.6406650618964) / Math.Sqrt((211.8055925427921 / (299)) + (32.555901037141275 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2431688319482067 = (525.402073603137 - 397.6406650618964) / 525.402073603137 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>.DataContractJsonSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.4665002372954765 < 4.344834083444449.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 106.90449145548041 (T) = (0 -3462.4514144425416) / Math.Sqrt((5804.232059045535 / (299)) + (1326.0580932993596 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25385013163923087 = (4640.423541250858 - 3462.4514144425416) / 4640.423541250858 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>.SystemTextJson_Reflection_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 400.41935893421334 < 534.7666706478677.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 71.99309670489399 (T) = (0 -404.8264120441505) / Math.Sqrt((258.1302513710895 / (299)) + (46.208285249991086 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2720854020427466 = (556.1454780275252 - 404.8264120441505) / 556.1454780275252 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToUtf8Bytes - Duration of single invocation 16.82 μs 12.76 μs 0.76 0.12 True
SerializeToStream - Duration of single invocation 17.98 μs 11.97 μs 0.67 0.12 True
SerializeToString - Duration of single invocation 18.22 μs 14.16 μs 0.78 0.14 True
SerializeObjectProperty - Duration of single invocation 18.25 μs 13.80 μs 0.76 0.12 True
SerializeToWriter - Duration of single invocation 18.14 μs 12.18 μs 0.67 0.20 False

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.756269643961254 < 16.736583356914007.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 71.30515119087256 (T) = (0 -12539.836100189541) / Math.Sqrt((231921.369404385 / (299)) + (70435.23901325515 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.30916006393867423 = (18151.57961434989 - 12539.836100189541) / 18151.57961434989 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.973509615384618 < 17.17945520069019.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 18.502007806651946 (T) = (0 -13259.026835387778) / Math.Sqrt((225289.76838372683 / (299)) + (888371.1351965922 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26835709765137433 = (18122.265374030638 - 13259.026835387778) / 18122.265374030638 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.155080475330925 < 17.35150972980018.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 46.8074899076723 (T) = (0 -13722.008452922506) / Math.Sqrt((278661.2463391457 / (299)) + (133021.21411457847 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26493560724970794 = (18667.763788122975 - 13722.008452922506) / 18667.763788122975 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeObjectProperty(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.797807502395166 < 17.408522831712375.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 34.51991608184673 (T) = (0 -14174.35241718149) / Math.Sqrt((269838.48943002906 / (299)) + (261106.4526269785 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2608009210443422 = (19175.284197062378 - 14174.35241718149) / 19175.284197062378 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToWriter(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.177555321053978 < 16.946959338319168.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 18.358483876731928 (T) = (0 -12971.271578066975) / Math.Sqrt((291950.96640275174 / (299)) + (806616.04154437 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2621610524680007 = (17580.08522246574 - 12971.271578066975) / 17580.08522246574 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToString - Duration of single invocation 25.25 μs 22.11 μs 0.88 0.08 False
SerializeObjectProperty - Duration of single invocation 26.17 μs 20.32 μs 0.78 0.07 False
SerializeToStream - Duration of single invocation 24.44 μs 20.85 μs 0.85 0.08 False
SerializeToWriter - Duration of single invocation 23.95 μs 20.66 μs 0.86 0.08 False
SerializeToUtf8Bytes - Duration of single invocation 23.78 μs 20.10 μs 0.85 0.10 False

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 22.108015993907085 < 24.5594397573002.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 21.023617240934136 (T) = (0 -21412.138342421327) / Math.Sqrt((279492.9326880817 / (299)) + (617218.8591280226 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.177656888773337 = (26037.961588176404 - 21412.138342421327) / 26037.961588176404 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeObjectProperty(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.323768233606668 < 24.461339121077508.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 39.89012972516289 (T) = (0 -20907.844964852364) / Math.Sqrt((528759.4566218371 / (299)) + (229469.65505100528 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21003357812270732 = (26466.751479343293 - 20907.844964852364) / 26466.751479343293 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.852332554517133 < 23.563309454235377.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 28.06234531211427 (T) = (0 -20443.669526795646) / Math.Sqrt((544173.3901438684 / (299)) + (342878.1394761133 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18731588991359857 = (25155.739201818717 - 20443.669526795646) / 25155.739201818717 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToWriter(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.66080672043011 < 23.40177366996687.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 31.607929260798535 (T) = (0 -20103.378290090273) / Math.Sqrt((338520.16888540686 / (299)) + (299309.3097822179 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19637734710046928 = (25015.942765619882 - 20103.378290090273) / 25015.942765619882 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.1042955955335 < 23.568995918191003.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 44.52555056836399 (T) = (0 -20366.13619599199) / Math.Sqrt((555187.4511552826 / (299)) + (141260.91652303192 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19781916281269857 = (25388.460122535555 - 20366.13619599199) / 25388.460122535555 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ILogB - Duration of single invocation 11.14 μs 4.96 μs 0.45 0.00 True
Hypot - Duration of single invocation 16.72 μs 12.54 μs 0.75 0.03 False
Max - Duration of single invocation 6.97 μs 5.30 μs 0.76 0.01 False
ScaleB - Duration of single invocation 12.15 μs 5.76 μs 0.47 0.10 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Single.ILogB

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.9601498572787825 < 10.589024882608177.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 1238.372807424232 (T) = (0 -4936.340853966742) / Math.Sqrt((818.2896190397848 / (299)) + (291.76632883449923 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5572984968762829 = (11150.494902627954 - 4936.340853966742) / 11150.494902627954 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.MathBenchmarks.Single.Hypot

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.541620501750998 < 15.87945101038667.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/6/2023 1:55:21 PM, 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 52.1245476725519 (T) = (0 -13034.34384371302) / Math.Sqrt((449250.8358045284 / (299)) + (28870.415893438676 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19615165796096368 = (16214.92906317402 - 13034.34384371302) / 16214.92906317402 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.MathBenchmarks.Single.Max

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.3049900220788055 < 6.620195886299435.
IsChangePoint: Marked as a change because one of 5/1/2023 3:42:23 PM, 5/9/2023 1:22:51 PM, 5/11/2023 9:15:05 PM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 13.175442506292567 (T) = (0 -5304.415514173499) / Math.Sqrt((325079.87838221656 / (299)) + (8.619120803265107 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.0757113425607038 = (5738.91659437774 - 5304.415514173499) / 5738.91659437774 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.MathBenchmarks.Single.ScaleB

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.7595151710979815 < 10.981038082897832.
IsChangePoint: Marked as a change because one of 4/11/2023 3:40:00 PM, 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 88.77847291262721 (T) = (0 -5742.065565103909) / Math.Sqrt((1268723.8429101321 / (299)) + (1757.5615487809694 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5056976512019512 = (11616.504714303666 - 5742.065565103909) / 11616.504714303666 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 29.47 ns 20.45 ns 0.69 0.06 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int32>.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.44536630774138 < 27.927792905679684.
IsChangePoint: Marked as a change because one of 3/6/2023 11:36:59 AM, 3/17/2023 10:06:19 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 68.65302842749284 (T) = (0 -20.903604969818108) / Math.Sqrt((1.871093979852252 / (299)) + (0.14007731408293783 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.30002954064747106 = (29.86355308358855 - 20.903604969818108) / 29.86355308358855 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
XmlSerializer_ - Duration of single invocation 821.46 μs 660.08 μs 0.80 0.05 True
DataContractSerializer_BinaryXml_ - Duration of single invocation 681.70 μs 556.50 μs 0.82 0.05 True
DataContractSerializer_ - Duration of single invocation 1.60 ms 1.36 ms 0.85 0.03 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>.XmlSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 660.0826385224275 < 778.8612792191577.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 29.004141886293887 (T) = (0 -673924.5517447108) / Math.Sqrt((200388393.4492697 / (299)) + (271277944.73874384 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1664808497703997 = (808529.1760352143 - 673924.5517447108) / 808529.1760352143 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>.DataContractSerializer_BinaryXml_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 556.4973841554558 < 661.9103698514913.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 48.84719333081359 (T) = (0 -556150.9774102719) / Math.Sqrt((195575157.05395332 / (299)) + (108234647.16964127 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20835794198371538 = (702528.335601431 - 556150.9774102719) / 702528.335601431 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>.DataContractSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3607342440801458 < 1.5198620064102561.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 46.99868674589066 (T) = (0 -1339173.965875012) / Math.Sqrt((819944466.4396127 / (299)) + (419642091.86803865 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17197503896465582 = (1617311.1064194713 - 1339173.965875012) / 1617311.1064194713 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Environment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetEnvironmentVariable - Duration of single invocation 162.09 ns 149.57 ns 0.92 0.13 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Environment*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Environment.GetEnvironmentVariable

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 149.56611667543532 < 157.50932860098396.
IsChangePoint: Marked as a change because one of 3/16/2023 10:20:48 AM, 4/10/2023 1:33:42 PM, 4/18/2023 3:11:52 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 25.091117867225226 (T) = (0 -150.93414748886477) / Math.Sqrt((30.124290424295406 / (299)) + (1.2423401818005233 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06860310493441145 = (162.0513749707487 - 150.93414748886477) / 162.0513749707487 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Booleans

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteBooleans - Duration of single invocation 1.55 ms 804.98 μs 0.52 0.04 True
WriteBooleans - Duration of single invocation 1.67 ms 995.38 μs 0.60 0.04 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Booleans*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Booleans.WriteBooleans(Formatted: False, SkipValidation: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 804.9818750000002 < 1.4779063705565267.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 21.82423307953249 (T) = (0 -904413.5330917748) / Math.Sqrt((1423223652.999935 / (299)) + (12074223051.797758 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4243917246656392 = (1571230.9427212747 - 904413.5330917748) / 1571230.9427212747 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Booleans.WriteBooleans(Formatted: False, SkipValidation: False)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 995.3795416666667 < 1.5908559662946427.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 38.33588337731813 (T) = (0 -876302.6686453051) / Math.Sqrt((719768665.5849252 / (299)) + (6011339146.764472 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.48537860389832627 = (1702810.4064141437 - 876302.6686453051) / 1702810.4064141437 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Xml.Tests.Perf_XmlConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DateTime_ToString_RoundtripKind - Duration of single invocation 70.24 ns 60.18 ns 0.86 0.05 False
DateTime_ToString - Duration of single invocation 73.27 ns 61.85 ns 0.84 0.03 False
DateTime_ToString_Unspecified - Duration of single invocation 75.25 ns 60.37 ns 0.80 0.03 False
DateTime_ToString_Local - Duration of single invocation 184.70 ns 145.38 ns 0.79 0.06 True

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Tests.Perf_XmlConvert*'

Payloads

Baseline
Compare

Histogram

System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_RoundtripKind

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 60.1789139906923 < 67.23303161756488.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/19/2023 8:38:27 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 14.621904402278055 (T) = (0 -60.70086598547737) / Math.Sqrt((4.252290051682499 / (299)) + (2.600643604879861 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10031823579710464 = (67.46926346701873 - 60.70086598547737) / 67.46926346701873 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Xml.Tests.Perf_XmlConvert.DateTime_ToString

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 61.85238827969258 < 70.00684889370199.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/25/2023 12:18:49 AM, 4/9/2023 11:04:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 60.340371618698086 (T) = (0 -61.990554740965266) / Math.Sqrt((11.130330334097636 / (299)) + (0.030923367443109143 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16227541463906772 = (73.99872920556192 - 61.990554740965266) / 73.99872920556192 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_Unspecified

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 60.37340672493343 < 71.5653356767249.
IsChangePoint: Marked as a change because one of 3/17/2023 9:25:32 PM, 4/12/2023 8:39:35 PM, 5/11/2023 5:02:01 PM, 5/16/2023 12:30:38 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 53.321632836229746 (T) = (0 -60.38508134847265) / Math.Sqrt((6.34922817196403 / (299)) + (0.008428766822710822 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11553422681099315 = (68.27294303401902 - 60.38508134847265) / 68.27294303401902 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_Local

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 145.3790269068956 < 175.6774355692159.
IsChangePoint: Marked as a change because one of 5/2/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 11.50172584214952 (T) = (0 -149.56602586612033) / Math.Sqrt((98.67118508255162 / (299)) + (157.395612542703 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21334344528077712 = (190.12874801444212 - 149.56602586612033) / 190.12874801444212 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 17.25 ns 11.88 ns 0.69 0.04 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.880181214486111 < 16.18350134835356.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 168.36837672405412 (T) = (0 -11.933946574255119) / Math.Sqrt((0.15706587280478232 / (299)) + (0.004797197102089736 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.29672198514747017 = (16.969031197082913 - 11.933946574255119) / 16.969031197082913 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromStream - Duration of single invocation 371.74 ns 338.18 ns 0.91 0.10 False
DeserializeFromReader - Duration of single invocation 262.64 ns 233.90 ns 0.89 0.04 False
DeserializeFromUtf8Bytes - Duration of single invocation 149.11 ns 127.04 ns 0.85 0.02 False
DeserializeFromString - Duration of single invocation 196.80 ns 162.03 ns 0.82 0.04 False
DeserializeFromStream - Duration of single invocation 366.75 ns 325.77 ns 0.89 0.09 False
DeserializeFromReader - Duration of single invocation 262.03 ns 231.99 ns 0.89 0.05 False
DeserializeFromUtf8Bytes - Duration of single invocation 147.08 ns 132.74 ns 0.90 0.04 False
DeserializeFromString - Duration of single invocation 196.24 ns 157.61 ns 0.80 0.04 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 338.1823174679382 < 360.1698265259378.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 37.62699060629697 (T) = (0 -329.2852503024548) / Math.Sqrt((141.2342953493078 / (299)) + (15.655650158503112 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1288901067978259 = (378.0065556275707 - 329.2852503024548) / 378.0065556275707 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromReader(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 233.90402297982845 < 251.96879004913902.
IsChangePoint: Marked as a change because one of 3/13/2023 11:01:07 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 37.993395420756215 (T) = (0 -234.21773142838597) / Math.Sqrt((38.002832207536706 / (299)) + (6.871759609707856 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11610253767274609 = (264.9829209959529 - 234.21773142838597) / 264.9829209959529 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 127.0424173738646 < 138.3011581459561.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 19.122763385577393 (T) = (0 -130.3133820356054) / Math.Sqrt((13.262600351771185 / (299)) + (8.056115448972657 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10680922771563106 = (145.89647148091782 - 130.3133820356054) / 145.89647148091782 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 162.02529948386825 < 192.33847573636427.
IsChangePoint: Marked as a change because one of 4/25/2023 1:57:48 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 75.04379353920801 (T) = (0 -159.87724784270742) / Math.Sqrt((29.230574991054464 / (299)) + (1.5876837411896636 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18039894577628446 = (195.06715739175692 - 159.87724784270742) / 195.06715739175692 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromStream(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 325.7746205330465 < 357.1279518349117.
IsChangePoint: Marked as a change because one of 4/19/2023 8:38:27 PM, 5/11/2023 9:15:05 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 27.025525656009812 (T) = (0 -328.5186236977213) / Math.Sqrt((118.05407478484184 / (299)) + (38.18238410422733 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13055790006308368 = (377.849915160029 - 328.5186236977213) / 377.849915160029 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromReader(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 231.98668433817906 < 249.194608959232.
IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 52.348180328608024 (T) = (0 -232.84182195938476) / Math.Sqrt((49.74597600440029 / (299)) + (2.6966854902031385 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12084595303906094 = (264.8475801985701 - 232.84182195938476) / 264.8475801985701 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromUtf8Bytes(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 132.73500420335637 < 138.84457989660177.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 16.127789686214868 (T) = (0 -131.68269102922292) / Math.Sqrt((10.16512260935749 / (299)) + (10.382313033511629 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10052268981794055 = (146.39912484570576 - 131.68269102922292) / 146.39912484570576 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromString(Mode: Reflection)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 157.61003562025707 < 187.06489821181788.
IsChangePoint: Marked as a change because one of 3/20/2023 4:51:16 PM, 4/19/2023 3:30:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 57.503400227376495 (T) = (0 -158.26769875672093) / Math.Sqrt((76.24175557481888 / (299)) + (2.1101941142489555 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1900932072158708 = (195.41470718212048 - 158.26769875672093) / 195.41470718212048 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Binary_FromStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinaryFormatter_ - Duration of single invocation 1.22 ms 877.56 μs 0.72 0.03 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_FromStream&lt;CollectionsOfPrimitives&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_FromStream<CollectionsOfPrimitives>.BinaryFormatter_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 877.5607953216374 < 1.1575309694608886.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 88.75481028192993 (T) = (0 -871736.4130442721) / Math.Sqrt((286163912.5770908 / (299)) + (186527763.00653473 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.284854408329806 = (1218963.5553906814 - 871736.4130442721) / 1218963.5553906814 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_FromString<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SystemTextJson_SourceGen_ - Duration of single invocation 504.86 ns 381.81 ns 0.76 0.06 True
JsonNet_ - Duration of single invocation 1.30 μs 740.91 ns 0.57 0.03 True
SystemTextJson_Reflection_ - Duration of single invocation 532.13 ns 397.01 ns 0.75 0.06 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;LoginViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_FromString<LoginViewModel>.SystemTextJson_SourceGen_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 381.8063220679714 < 473.1502131668493.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 76.93708827000077 (T) = (0 -375.01582660155316) / Math.Sqrt((178.14922900268238 / (299)) + (22.102395063375216 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23714509001719006 = (491.595219083015 - 375.01582660155316) / 491.595219083015 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromString<LoginViewModel>.JsonNet_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 740.9053500381272 < 1.2221911895427315.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 99.15302237584618 (T) = (0 -741.6311472660333) / Math.Sqrt((718.203694106729 / (299)) + (392.00774862280997 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.43273531537073884 = (1307.381135052908 - 741.6311472660333) / 1307.381135052908 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromString<LoginViewModel>.SystemTextJson_Reflection_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 397.0145465464047 < 495.19466822647314.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 70.59289717663721 (T) = (0 -391.71476548575754) / Math.Sqrt((276.97938656932337 / (299)) + (27.638546815612397 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2394602008018956 = (515.0483457917292 - 391.71476548575754) / 515.0483457917292 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Enumerator - Duration of single invocation 156.60 ns 92.15 ns 0.59 0.21 True
GetCount - Duration of single invocation 9.05 ns 3.97 ns 0.44 0.05 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList*'

Payloads

Baseline
Compare

Histogram

XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList.Enumerator

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 92.15221071433774 < 155.73386552094024.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 62.09290086437309 (T) = (0 -92.86612972889222) / Math.Sqrt((43.80140363561947 / (299)) + (14.777742645846132 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.43098529227954296 = (163.20514824814524 - 92.86612972889222) / 163.20514824814524 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList.GetCount

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.9696157214436325 < 8.601260602981242.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 249.76379282348262 (T) = (0 -3.9680168602134653) / Math.Sqrt((0.08957753097370136 / (299)) + (0.0017441845635690718 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5672740319063506 = (9.169814507999941 - 3.9680168602134653) / 9.169814507999941 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_FromStream<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractSerializer_ - Duration of single invocation 2.26 μs 1.73 μs 0.77 0.03 True
XmlSerializer_ - Duration of single invocation 3.87 μs 3.26 μs 0.84 0.00 True
DataContractSerializer_BinaryXml_ - Duration of single invocation 761.01 ns 513.70 ns 0.68 0.04 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;SimpleStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_FromStream<SimpleStructWithProperties>.DataContractSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.7267516176520934 < 2.103256593040305.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 31.49392867261946 (T) = (0 -1750.590321812157) / Math.Sqrt((1276.1474846249662 / (299)) + (3312.2513982985115 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22454214382679044 = (2257.492535379173 - 1750.590321812157) / 2257.492535379173 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_FromStream<SimpleStructWithProperties>.XmlSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.261461163266377 < 3.691689677014097.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 51.376083100045456 (T) = (0 -3292.2572382092912) / Math.Sqrt((1109.961465674704 / (299)) + (1592.402180329028 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14916016203599738 = (3869.420649234551 - 3292.2572382092912) / 3869.420649234551 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_FromStream<SimpleStructWithProperties>.DataContractSerializer_BinaryXml_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 513.7049065456347 < 683.7688280453924.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 105.81538951936126 (T) = (0 -499.2846814342928) / Math.Sqrt((206.1757322411491 / (299)) + (46.71937487567487 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.30489130816095733 = (718.2828920083562 - 499.2846814342928) / 718.2828920083562 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedSet - Duration of single invocation 24.41 μs 9.91 μs 0.41 0.02 True
ImmutableSortedSet - Duration of single invocation 24.53 μs 11.35 μs 0.46 0.06 True
Stack - Duration of single invocation 12.29 μs 11.37 μs 0.93 0.00 True
ImmutableHashSet - Duration of single invocation 10.19 μs 5.90 μs 0.58 0.26 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<Int32>.SortedSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.90560919630093 < 22.908716702994383.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 216.1669092973012 (T) = (0 -10040.718812808173) / Math.Sqrt((505102.2238683851 / (299)) + (21961.766856397328 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5558326428588827 = (22605.710778557088 - 10040.718812808173) / 22605.710778557088 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<Int32>.ImmutableSortedSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.350956928503109 < 21.375774203114215.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 247.2106978363541 (T) = (0 -11357.849779655116) / Math.Sqrt((402454.6188457418 / (299)) + (3459.9344693285034 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4663606095647669 = (21283.754503938926 - 11357.849779655116) / 21283.754503938926 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<Int32>.Stack(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.373752885082604 < 11.730228276737614.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 112.9175793531448 (T) = (0 -11347.650136472328) / Math.Sqrt((10866.092214989314 / (299)) + (528.4480252861989 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08030120254827325 = (12338.441855000843 - 11347.650136472328) / 12338.441855000843 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<Int32>.ImmutableHashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.900825810107642 < 9.902793285148736.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 25.586010618153754 (T) = (0 -6620.308461370063) / Math.Sqrt((179949.61889313758 / (299)) + (310945.7835433266 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3770204830078462 = (10626.84772259285 - 6620.308461370063) / 10626.84772259285 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CtorFromCollectionNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 84.16 μs 48.74 μs 0.58 0.08 True
Hashtable - Duration of single invocation 22.39 μs 16.27 μs 0.73 0.02 True
Stack - Duration of single invocation 25.36 μs 21.64 μs 0.85 0.12 False
Queue - Duration of single invocation 25.41 μs 21.54 μs 0.85 0.12 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollectionNonGeneric&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollectionNonGeneric<Int32>.SortedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 48.74410861713107 < 79.09421924306005.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 91.37739755705233 (T) = (0 -49736.148379799706) / Math.Sqrt((7314109.148689807 / (299)) + (1276458.6285821043 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3915170317219844 = (81737.94661919819 - 49736.148379799706) / 81737.94661919819 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollectionNonGeneric<Int32>.Hashtable(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.272494524057024 < 21.179309056415978.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 44.80874701980192 (T) = (0 -16362.27656857362) / Math.Sqrt((295736.2577395398 / (299)) + (234454.99917291384 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2741635553988881 = (22542.649504966113 - 16362.27656857362) / 22542.649504966113 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollectionNonGeneric<Int32>.Stack(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.641086951122592 < 25.191993490860852.
IsChangePoint: Marked as a change because one of 3/17/2023 5:15:23 PM, 4/4/2023 4:23:54 PM, 4/19/2023 8:38:27 PM, 5/20/2023 9:58:12 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 39.89599262738297 (T) = (0 -21862.569486500615) / Math.Sqrt((1917646.6937523016 / (299)) + (56655.870580406954 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1955513543450469 = (27177.08532991674 - 21862.569486500615) / 27177.08532991674 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollectionNonGeneric<Int32>.Queue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.542260060015742 < 24.79634566968387.
IsChangePoint: Marked as a change because one of 3/8/2023 9:36:36 PM, 4/26/2023 6:27:29 AM, 5/20/2023 9:58:12 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 55.35227502570576 (T) = (0 -21622.61595792231) / Math.Sqrt((1451822.8091101954 / (299)) + (23874.51560245636 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.20077907709959997 = (27054.616988070204 - 21622.61595792231) / 27054.616988070204 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchmark.GetChildKeysTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AddChainedConfigurationWithSplitting - Duration of single invocation 368.52 μs 304.49 μs 0.83 0.15 False
AddChainedConfigurationWithCommonPaths - Duration of single invocation 375.47 μs 257.90 μs 0.69 0.18 False
AddChainedConfigurationNoDelimiter - Duration of single invocation 712.15 μs 579.57 μs 0.81 0.11 False
AddChainedConfigurationEmpty - Duration of single invocation 33.71 ms 14.02 ms 0.42 0.03 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchmark.GetChildKeysTests*'

Payloads

Baseline
Compare

Histogram

Benchmark.GetChildKeysTests.AddChainedConfigurationWithSplitting

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 304.4919471153846 < 385.37904801968864.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 32.286730527620485 (T) = (0 -284194.54253500886) / Math.Sqrt((902443508.6698554 / (299)) + (104836399.49826773 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27441890573829075 = (391678.53846051695 - 284194.54253500886) / 391678.53846051695 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Benchmark.GetChildKeysTests.AddChainedConfigurationWithCommonPaths

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 257.89886612021854 < 389.75921254006414.
IsChangePoint: Marked as a change because one of 4/18/2023 7:42:00 PM, 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 24.921242978496036 (T) = (0 -276338.8152735166) / Math.Sqrt((802407214.8225348 / (299)) + (241323212.58207265 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2936338357568076 = (391211.85195724753 - 276338.8152735166) / 391211.85195724753 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Benchmark.GetChildKeysTests.AddChainedConfigurationNoDelimiter

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 579.5720200892857 < 723.8810969387755.
IsChangePoint: Marked as a change because one of 4/18/2023 7:42:00 PM, 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 28.253442280890606 (T) = (0 -560754.3727612428) / Math.Sqrt((1604646120.7430418 / (299)) + (541558839.7636839 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2567886446718462 = (754501.8906683014 - 560754.3727612428) / 754501.8906683014 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Benchmark.GetChildKeysTests.AddChainedConfigurationEmpty

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.02218375 < 32.00834719436813.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 5/2/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 25.650302966222416 (T) = (0 -14035231.73076923) / Math.Sqrt((90412314064939.83 / (299)) + (200668734447.79648 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5074612119314202 = (28495688.199108902 - 14035231.73076923) / 28495688.199108902 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IterateTryGetMemory - Duration of single invocation 17.54 ns 15.45 ns 0.88 0.16 False
IterateGetPositionMemory - Duration of single invocation 34.52 ns 32.12 ns 0.93 0.04 False
IterateForEachMemory - Duration of single invocation 30.27 ns 26.08 ns 0.86 0.10 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateTryGetMemory

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 15.446855136886711 < 16.84466977351401.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 41.23508458473487 (T) = (0 -15.559289700298525) / Math.Sqrt((1.2082706002166819 / (299)) + (0.03162596905363093 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17575686250005249 = (18.877063080551903 - 15.559289700298525) / 18.877063080551903 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateGetPositionMemory

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 32.1212103875158 < 32.69448883881964.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 20.040108980326387 (T) = (0 -31.154845490844156) / Math.Sqrt((0.4823792441896534 / (299)) + (0.3066128372289365 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09264899085772263 = (34.336045452018574 - 31.154845490844156) / 34.336045452018574 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachMemory

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 26.08010078510327 < 28.47606410057409.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 10.05040924802283 (T) = (0 -27.18135328232645) / Math.Sqrt((0.40353586467745467 / (299)) + (0.8099928577024582 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08532957401295915 = (29.717100837708248 - 27.18135328232645) / 29.717100837708248 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CtorFromCollectionNonGeneric<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Stack - Duration of single invocation 5.21 μs 2.51 μs 0.48 0.06 False
Hashtable - Duration of single invocation 29.09 μs 24.63 μs 0.85 0.02 False
Queue - Duration of single invocation 5.55 μs 3.18 μs 0.57 0.12 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollectionNonGeneric&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollectionNonGeneric<String>.Stack(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.5109400128658734 < 4.997816278752928.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 216.51645029344047 (T) = (0 -2533.886200352782) / Math.Sqrt((13131.577930482228 / (299)) + (1449.0951010831488 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5157740208626841 = (5232.8588500499 - 2533.886200352782) / 5232.8588500499 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollectionNonGeneric<String>.Hashtable(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 24.62501616985377 < 27.50288177969471.
IsChangePoint: Marked as a change because one of 3/25/2023 10:49:06 AM, 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 57.28668078443479 (T) = (0 -24555.954451737165) / Math.Sqrt((438440.50553007657 / (299)) + (48398.19352483513 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14387580221669355 = (28682.70107925687 - 24555.954451737165) / 28682.70107925687 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollectionNonGeneric<String>.Queue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.175496277915633 < 5.280821320277445.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 83.88717518216826 (T) = (0 -3213.5601060318145) / Math.Sqrt((27456.351017060802 / (299)) + (8414.049408017036 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.41508793444127323 = (5494.09098436381 - 3213.5601060318145) / 5494.09098436381 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Object>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultipleSerial - Duration of single invocation 820.85 ns 641.12 ns 0.78 0.13 False
MultipleSerial - Duration of single invocation 500.85 ns 463.97 ns 0.93 0.25 False
MultipleSerial - Duration of single invocation 509.08 ns 427.56 ns 0.84 0.18 False
SingleSerial - Duration of single invocation 25.36 ns 22.61 ns 0.89 0.07 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Object>.MultipleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 641.117171875 < 672.0779963141024.
IsChangePoint: Marked as a change because one of 5/11/2023 9:15:05 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 17.417137196208998 (T) = (0 -639.9744659365347) / Math.Sqrt((783.2663898918279 / (299)) + (794.3902778267275 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1784810003182496 = (779.0135909022864 - 639.9744659365347) / 779.0135909022864 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.RentReturnArrayPoolTests<Object>.MultipleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 463.96799999999996 < 497.95909156249996.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.045145414458888 (T) = (0 -471.27794504551815) / Math.Sqrt((379.2944986914516 / (299)) + (200.83635661420388 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09461378110863869 = (520.5269698301731 - 471.27794504551815) / 520.5269698301731 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.RentReturnArrayPoolTests<Object>.MultipleSerial(RentalSize: 4096, ManipulateArray: False, Async: False, UseSharedPool: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 427.5550875 < 459.29903234374996.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/8/2023 11:50:12 PM, 5/16/2023 11:15:26 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 18.912221503884137 (T) = (0 -428.1137398294806) / Math.Sqrt((1168.7025496840256 / (299)) + (737.8362366572898 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.20949364863739806 = (541.5690071199778 - 428.1137398294806) / 541.5690071199778 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: False, UseSharedPool: False)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 22.61491185897436 < 24.09585216517857.
IsChangePoint: Marked as a change because one of 4/18/2023 7:36:21 AM, 4/19/2023 1:39:34 AM, 5/20/2023 9:58:12 PM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 34.33273588959235 (T) = (0 -22.72098733974359) / Math.Sqrt((1.0615273693541327 / (299)) + (0.009521169540617373 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.10039461956540428 = (25.256615660487906 - 22.72098733974359) / 25.256615660487906 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_PriorityQueue<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Enumerate - Duration of single invocation 5.31 μs 4.48 μs 0.84 0.08 False
Enumerate - Duration of single invocation 523.64 ns 471.08 ns 0.90 0.10 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_PriorityQueue<String, String>.Enumerate(Size: 1000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.479745111397973 < 5.049021916262817.
IsChangePoint: Marked as a change because one of 3/6/2023 11:36:59 AM, 3/31/2023 8:05:26 PM, 4/3/2023 9:14:01 PM, 4/19/2023 1:39:34 AM, 4/24/2023 11:08:13 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 55.16952076667761 (T) = (0 -4479.888326214777) / Math.Sqrt((72461.51916170448 / (299)) + (2.0098305702855686 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1609143959831307 = (5339.012259021801 - 4479.888326214777) / 5339.012259021801 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<String, String>.Enumerate(Size: 100)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 471.08219255631315 < 497.58683116010656.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 48.35641425677684 (T) = (0 -470.06650257548006) / Math.Sqrt((619.5798209635826 / (299)) + (2.4786605674585336 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13400928705055223 = (542.8077871349184 - 470.06650257548006) / 542.8077871349184 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Baseline - Duration of single invocation 1.46 μs 1.01 μs 0.69 0.13 False
CaseInsensitiveNotMatching - Duration of single invocation 1.38 μs 958.03 ns 0.69 0.08 False
MissingProperties - Duration of single invocation 933.42 ns 614.34 ns 0.66 0.14 False
CaseInsensitiveMatching - Duration of single invocation 1.42 μs 1.03 μs 0.73 0.06 True

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>.Baseline

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0131752647643713 < 1.3541149053866344.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 53.3110321912802 (T) = (0 -994.7990338528859) / Math.Sqrt((2552.981320409621 / (299)) + (496.65031622937187 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2681413738843959 = (1359.277595910645 - 994.7990338528859) / 1359.277595910645 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>.CaseInsensitiveNotMatching

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 958.027505009308 < 1.325864539623365.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 53.18980992405814 (T) = (0 -989.4343156117104) / Math.Sqrt((1956.492349073806 / (299)) + (528.8522661252424 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2697657797642825 = (1354.9547367039631 - 989.4343156117104) / 1354.9547367039631 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>.MissingProperties

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 614.3445600085594 < 852.8628426130642.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 66.66332525559996 (T) = (0 -609.1527892148429) / Math.Sqrt((1238.907646881953 / (299)) + (173.47947521086556 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.31396342464974564 = (887.9304851987539 - 609.1527892148429) / 887.9304851987539 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>.CaseInsensitiveMatching

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0330345922313031 < 1.3343459847522932.
IsChangePoint: Marked as a change because one of 4/25/2023 9:59:52 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 54.935976373277114 (T) = (0 -984.7943460356358) / Math.Sqrt((1396.276526511337 / (299)) + (528.7232209326098 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27305842491491766 = (1354.7090712487782 - 984.7943460356358) / 1354.7090712487782 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Concurrent.AddRemoveFromSameThreads<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentBag - Duration of single invocation 328.95 ms 264.24 ms 0.80 0.67 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.AddRemoveFromSameThreads&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.AddRemoveFromSameThreads<Int32>.ConcurrentBag(Size: 2000000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 264.2433111111111 < 293.77561075.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 7.016029550950316 (T) = (0 -246887085.47008547) / Math.Sqrt((806893897244319.1 / (299)) + (774318621976196.9 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18316330077425735 = (302247787.96557873 - 246887085.47008547) / 302247787.96557873 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.MDBenchI.MDMidpoint

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 528.45 ms 334.19 ms 0.63 0.02 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDMidpoint*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDMidpoint.Test

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 334.19194285714286 < 500.51384599999994.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 85.89142869414988 (T) = (0 -335569318.9264581) / Math.Sqrt((46488581629526.33 / (299)) + (64136505398888.086 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.36605045744165116 = (529331273.8617084 - 335569318.9264581) / 529331273.8617084 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in BenchmarksGame.Fasta_2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
RunBench - Duration of single invocation 460.72 μs 336.63 μs 0.73 0.02 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BenchmarksGame.Fasta_2*'

Payloads

Baseline
Compare

Histogram

BenchmarksGame.Fasta_2.RunBench

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 336.6346808510639 < 437.2849912683823.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 252.24863152714298 (T) = (0 -336912.6499209788) / Math.Sqrt((44011810.316586874 / (299)) + (1385915.7705195115 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2738846603346588 = (463993.2962664833 - 336912.6499209788) / 463993.2962664833 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse_SequentialAsync - Duration of single invocation 477.92 ns 380.90 ns 0.80 0.03 True
Parse_SequentialAsync - Duration of single invocation 476.50 ns 383.98 ns 0.81 0.03 True
Parse_ParallelAsync - Duration of single invocation 1.04 μs 967.47 ns 0.93 0.44 False
Parse_SequentialAsync - Duration of single invocation 211.72 ns 174.73 ns 0.83 0.05 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark.Parse_SequentialAsync(Length: 4096, Chunks: 16)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 380.9049306933421 < 443.34511115884715.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 34.53756909326634 (T) = (0 -382.221583967805) / Math.Sqrt((59.678907745460634 / (299)) + (103.51108004445348 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20518309307280921 = (480.89261896239253 - 382.221583967805) / 480.89261896239253 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark.Parse_SequentialAsync(Length: 128, Chunks: 16)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 383.9805772569445 < 443.34852074872833.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 85.00228628313158 (T) = (0 -381.09604855069244) / Math.Sqrt((64.69064631738866 / (299)) + (15.240542936168339 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20813779635033994 = (481.2656126207774 - 381.09604855069244) / 481.2656126207774 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark.Parse_ParallelAsync(Length: 128, Chunks: 16)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 967.467403927365 < 1.0599740551201582.
IsChangePoint: Marked as a change because one of 4/3/2023 9:07:40 AM, 4/12/2023 4:44:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 6.9378159465194935 (T) = (0 -945.5651579011111) / Math.Sqrt((5362.604905320074 / (299)) + (9730.801907849205 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16883511633966417 = (1137.6384836387363 - 945.5651579011111) / 1137.6384836387363 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark.Parse_SequentialAsync(Length: 128, Chunks: 1)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 174.72787832284902 < 198.24064777218646.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 24.115966394063925 (T) = (0 -175.90446582522492) / Math.Sqrt((18.96802463482641 / (299)) + (25.580159007672382 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16345157628827142 = (210.2740987123549 - 175.90446582522492) / 210.2740987123549 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SystemTextJson_SourceGen_ - Duration of single invocation 308.08 μs 227.42 μs 0.74 0.05 True
SystemTextJson_Reflection_ - Duration of single invocation 300.94 μs 230.80 μs 0.77 0.07 True
JsonNet_ - Duration of single invocation 742.40 μs 491.13 μs 0.66 0.07 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>.SystemTextJson_SourceGen_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 227.4163212121212 < 290.8110858958043.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 50.19623619388383 (T) = (0 -227058.19688609164) / Math.Sqrt((35092706.24541124 / (299)) + (28434202.99653454 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25127762248940894 = (303260.86638552457 - 227058.19688609164) / 303260.86638552457 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>.SystemTextJson_Reflection_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 230.80135052179253 < 294.9674218965141.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 45.5218787565287 (T) = (0 -232451.48921279548) / Math.Sqrt((65756555.807688564 / (299)) + (32678742.24460577 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24459212715026132 = (307716.5297945384 - 232451.48921279548) / 307716.5297945384 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>.JsonNet_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 491.1344066366704 < 704.0338048394519.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 102.84671027148914 (T) = (0 -496032.20476368733) / Math.Sqrt((210295300.89297658 / (299)) + (63512339.98871932 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.32893395542192844 = (739170.4717760893 - 496032.20476368733) / 739170.4717760893 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.TreeSort

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 417.66 μs 385.47 μs 0.92 0.00 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.TreeSort*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.TreeSort.Test

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 385.4728229166667 < 396.54640560031186.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 25.950339737489276 (T) = (0 -386592.86103168444) / Math.Sqrt((40642131.94239519 / (299)) + (8624940.283263573 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.05661799420574562 = (409794.609879381 - 386592.86103168444) / 409794.609879381 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 13.22 μs 7.07 μs 0.54 0.38 True
EnumerateArray - Duration of single invocation 519.99 ns 472.00 ns 0.91 0.01 False
EnumerateArray - Duration of single invocation 520.07 ns 471.92 ns 0.91 0.01 False
Parse - Duration of single invocation 8.99 μs 4.60 μs 0.51 0.21 True
Parse - Duration of single invocation 733.76 μs 497.50 μs 0.68 0.14 True

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_EnumerateArray.Parse(TestCase: ArrayOfNumbers)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.073053298320791 < 13.079637286206031.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 101.26614617675304 (T) = (0 -6810.393332571645) / Math.Sqrt((657966.2699312181 / (299)) + (36768.34054909435 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5132529662651711 = (13991.6483523592 - 6810.393332571645) / 13991.6483523592 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfNumbers)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 472.0046869311977 < 493.21924596194305.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 42.02604909779404 (T) = (0 -471.9581847983987) / Math.Sqrt((147.8097416515553 / (299)) + (0.4985827356738533 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.061025319199789485 = (502.63142814052 - 471.9581847983987) / 502.63142814052 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfStrings)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 471.9165455767855 < 493.7484543058117.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 36.96338521634773 (T) = (0 -472.80067127589274) / Math.Sqrt((158.72149368244322 / (299)) + (1.283379305013389 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.058408448146273176 = (502.12926225291875 - 472.80067127589274) / 502.12926225291875 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_EnumerateArray.Parse(TestCase: ArrayOfStrings)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.603422521153281 < 8.694222962708736.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 164.75241611238022 (T) = (0 -4681.710997389637) / Math.Sqrt((129166.1658602546 / (299)) + (4558.229022717645 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4960883145336735 = (9290.737112113446 - 4681.710997389637) / 9290.737112113446 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_EnumerateArray.Parse(TestCase: Json400KB)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 497.4979296875 < 693.5341136363636.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 17.825618581316235 (T) = (0 -518392.91788698226) / Math.Sqrt((497285438.9569591 / (299)) + (1883507006.4406316 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.29392035250256865 = (734184.7619094107 - 518392.91788698226) / 734184.7619094107 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.MDBenchI.MDMulMatrix

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 851.27 ms 546.70 ms 0.64 0.00 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDMulMatrix*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDMulMatrix.Test

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 546.6988538461538 < 807.980472.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 1982.1213853369943 (T) = (0 -546486772.5500143) / Math.Sqrt((6535689117590.879 / (299)) + (19186180348.266937 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.35651979979173665 = (849267424.8145366 - 546486772.5500143) / 849267424.8145366 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 129.82 ns 93.75 ns 0.72 0.10 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Byte>.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 93.74841936972864 < 123.11422861149443.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 99.571424481388 (T) = (0 -94.63026225582598) / Math.Sqrt((31.72846985647196 / (299)) + (0.5641516469446892 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2891958346589521 = (133.13127141063086 - 94.63026225582598) / 133.13127141063086 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Binary_ToStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinaryFormatter_ - Duration of single invocation 38.50 μs 29.68 μs 0.77 0.03 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_ToStream&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_ToStream<MyEventsListerViewModel>.BinaryFormatter_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 29.676170548134134 < 36.738316851820315.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 17.380586589138797 (T) = (0 -30184.12499197452) / Math.Sqrt((154627.39451701732 / (299)) + (3129691.405527501 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22047562315277103 = (38721.207300859045 - 30184.12499197452) / 38721.207300859045 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_FromStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
JsonNet_ - Duration of single invocation 3.14 μs 1.83 μs 0.58 0.08 True
SystemTextJson_SourceGen_ - Duration of single invocation 1.46 μs 1.04 μs 0.72 0.08 True
SystemTextJson_Reflection_ - Duration of single invocation 1.49 μs 1.02 μs 0.68 0.08 True
DataContractJsonSerializer_ - Duration of single invocation 10.71 μs 7.92 μs 0.74 0.04 True

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_FromStream<Location>.JsonNet_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.8347159731412046 < 2.9808299831824234.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 80.47828645207575 (T) = (0 -1881.3914652285348) / Math.Sqrt((8891.065163981264 / (299)) + (3288.1669854002944 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4183291847314131 = (3234.4608253377833 - 1881.3914652285348) / 3234.4608253377833 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromStream<Location>.SystemTextJson_SourceGen_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0447074824238367 < 1.3879247514193127.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 49.967202599287106 (T) = (0 -1016.3351843612752) / Math.Sqrt((2182.110014507644 / (299)) + (727.1711234081841 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.28106834036732997 = (1413.6742633932133 - 1016.3351843612752) / 1413.6742633932133 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromStream<Location>.SystemTextJson_Reflection_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.015006192152262 < 1.410711697611639.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 64.8273888042797 (T) = (0 -1037.7161274209786) / Math.Sqrt((1187.1164778292052 / (299)) + (489.57206371747225 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2872776644951244 = (1455.9893463783271 - 1037.7161274209786) / 1455.9893463783271 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromStream<Location>.DataContractJsonSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.924565263842227 < 10.167424802734326.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 108.20616706099831 (T) = (0 -7874.849379960519) / Math.Sqrt((21002.839666318247 / (299)) + (7978.7817043705645 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2643627525327845 = (10704.799691795746 - 7874.849379960519) / 10704.799691795746 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_SortedSet

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnumerateViewBetween - Duration of single invocation 2.33 μs 1.34 μs 0.57 0.09 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_SortedSet*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_SortedSet.EnumerateViewBetween

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3395452533119279 < 2.2183560305151753.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 50.18760276180373 (T) = (0 -1304.736034105759) / Math.Sqrt((3662.2945264691866 / (299)) + (5545.2303573021445 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.44621865982191716 = (2356.0491108028073 - 1304.736034105759) / 2356.0491108028073 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Base64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteByteArrayAsBase64_HeavyEscaping - Duration of single invocation 174.97 ns 156.22 ns 0.89 0.04 False
WriteByteArrayAsBase64_HeavyEscaping - Duration of single invocation 72.47 ns 54.35 ns 0.75 0.07 False
WriteByteArrayAsBase64_NoEscaping - Duration of single invocation 177.69 ns 156.37 ns 0.88 0.04 False
WriteByteArrayAsBase64_NoEscaping - Duration of single invocation 71.95 ns 53.31 ns 0.74 0.06 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Base64*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_HeavyEscaping(NumberOfBytes: 1000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 156.22230506904384 < 168.95775906260818.
IsChangePoint: Marked as a change because one of 5/2/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 16.249333731240686 (T) = (0 -156.06735698933235) / Math.Sqrt((17.427435458888088 / (299)) + (8.539116927702965 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08092286471333217 = (169.80876903292088 - 156.06735698933235) / 169.80876903292088 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_HeavyEscaping(NumberOfBytes: 100)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 54.34541022033406 < 76.57439796538188.
IsChangePoint: Marked as a change because one of 3/15/2023 5:33:41 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 48.73935415662916 (T) = (0 -54.04485889479065) / Math.Sqrt((14.57188091102295 / (299)) + (0.742351656648308 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22683948577798757 = (69.90121443174439 - 54.04485889479065) / 69.90121443174439 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_NoEscaping(NumberOfBytes: 1000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 156.3686829142214 < 168.88625394515327.
IsChangePoint: Marked as a change because one of 3/6/2023 8:17:12 PM, 4/9/2023 11:04:55 PM, 5/1/2023 3:42:23 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 16.118219240889687 (T) = (0 -157.25115843512555) / Math.Sqrt((23.689306731668015 / (299)) + (6.816978788749927 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0737607171121173 = (169.7737953251548 - 157.25115843512555) / 169.7737953251548 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_NoEscaping(NumberOfBytes: 100)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 53.314510436016604 < 76.48214584788107.
IsChangePoint: Marked as a change because one of 3/15/2023 5:33:41 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 55.278402342381085 (T) = (0 -54.08247214681917) / Math.Sqrt((18.248852544280822 / (299)) + (0.2994389137816124 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2286058439861582 = (70.11003612768972 - 54.08247214681917) / 70.11003612768972 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
XmlSerializer_ - Duration of single invocation 2.56 μs 2.17 μs 0.85 0.06 False
DataContractSerializer_ - Duration of single invocation 1.87 μs 1.24 μs 0.66 0.04 True
DataContractSerializer_BinaryXml_ - Duration of single invocation 1.25 μs 942.30 ns 0.75 0.14 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<Location>.XmlSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.173367968677602 < 2.4358792637209015.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 25.549630187324677 (T) = (0 -2223.300280655231) / Math.Sqrt((2703.9868826471034 / (299)) + (3043.0590116639855 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15195651798712062 = (2621.6819394426584 - 2223.300280655231) / 2621.6819394426584 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_ToStream<Location>.DataContractSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.2384435765605433 < 1.7552832183367524.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 71.36680072175012 (T) = (0 -1278.4317713203513) / Math.Sqrt((2446.0888473083546 / (299)) + (840.2514444711873 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3226562559358505 = (1887.419471315401 - 1278.4317713203513) / 1887.419471315401 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_ToStream<Location>.DataContractSerializer_BinaryXml_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 942.3027839971903 < 1.175388494832225.
IsChangePoint: Marked as a change because one of 4/3/2023 7:13:05 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 55.708657620492154 (T) = (0 -864.8591046148928) / Math.Sqrt((1954.2504452936435 / (299)) + (718.2071928653149 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.33612301605597433 = (1302.7400038435626 - 864.8591046148928) / 1302.7400038435626 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractSerializer_BinaryXml_ - Duration of single invocation 449.19 μs 323.74 μs 0.72 0.06 True
DataContractSerializer_ - Duration of single invocation 688.04 μs 525.65 μs 0.76 0.08 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>.DataContractSerializer_BinaryXml_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 323.7426419235758 < 424.4612100300716.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 98.68894915869878 (T) = (0 -326413.0632293896) / Math.Sqrt((198389817.42383152 / (299)) + (14137063.525262652 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.28575248543165754 = (457002.72884625755 - 326413.0632293896) / 457002.72884625755 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>.DataContractSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 525.6539982030549 < 648.0899604959546.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 77.68552000732745 (T) = (0 -528865.4898611441) / Math.Sqrt((203159082.3772679 / (299)) + (42634898.91151911 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22617079445049315 = (683439.5575514488 - 528865.4898611441) / 683439.5575514488 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_FromString<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
JsonNet_ - Duration of single invocation 540.88 μs 452.10 μs 0.84 0.09 True
SystemTextJson_SourceGen_ - Duration of single invocation 290.46 μs 210.17 μs 0.72 0.14 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;CollectionsOfPrimitives&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_FromString<CollectionsOfPrimitives>.JsonNet_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 452.0973381294964 < 535.8347174656845.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 27.703967744521243 (T) = (0 -440895.02145329013) / Math.Sqrt((144828800.88690516 / (299)) + (265263944.52772522 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22311366220064668 = (567515.4781356964 - 440895.02145329013) / 567515.4781356964 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromString<CollectionsOfPrimitives>.SystemTextJson_SourceGen_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 210.16535574229687 < 270.2325999832788.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 14.128565954787657 (T) = (0 -219110.0528891544) / Math.Sqrt((60523521.91229084 / (299)) + (286213261.8773805 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23309702969266405 = (285707.6597334682 - 219110.0528891544) / 285707.6597334682 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 10.51 ns 7.94 ns 0.76 0.01 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.940899728428631 < 9.986362974190083.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 234.291610319638 (T) = (0 -7.89760574871569) / Math.Sqrt((0.007722493502185497 / (299)) + (0.0012996668141051213 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2496662023932109 = (10.525456502033265 - 7.89760574871569) / 10.525456502033265 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToString<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SystemTextJson_Reflection_ - Duration of single invocation 773.50 ns 652.15 ns 0.84 0.05 False
JsonNet_ - Duration of single invocation 1.66 μs 914.09 ns 0.55 0.13 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToString<Location>.SystemTextJson_Reflection_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 652.1528962349846 < 758.4191258396103.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 55.07357389537082 (T) = (0 -653.8607787557528) / Math.Sqrt((385.89324952800547 / (299)) + (108.06697157080451 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20699040311676759 = (824.5307261420586 - 653.8607787557528) / 824.5307261420586 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToString<Location>.JsonNet_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 914.0878358186196 < 1.547617347307956.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 113.69066075123595 (T) = (0 -902.0161807866997) / Math.Sqrt((3564.936584604151 / (299)) + (320.5655637506479 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.43256922205970216 = (1589.6497261937477 - 902.0161807866997) / 1589.6497261937477 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchF.Bisect

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 303.96 ms 150.91 ms 0.50 0.00 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.Bisect*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchF.Bisect.Test

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 150.90610357142856 < 288.64783216666666.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 5772.630603227614 (T) = (0 -150918825.36630037) / Math.Sqrt((151209672202.044 / (299)) + (2557178841.26637 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5034135477243766 = (303912490.31202924 - 150918825.36630037) / 303912490.31202924 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CertProp - Duration of single invocation 26.84 ns 24.17 ns 0.90 0.04 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests*'

Payloads

Baseline
Compare

Histogram

System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests.CertProp

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 24.166056732126343 < 25.20598175484342.
IsChangePoint: Marked as a change because one of 5/2/2023 9:04:55 PM, 5/3/2023 6:00:09 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 54.0547604566293 (T) = (0 -24.36300772313878) / Math.Sqrt((0.3514505941755595 / (299)) + (0.008274712320665753 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08629407135523295 = (26.663948387939914 - 24.36300772313878) / 26.663948387939914 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_VectorOf<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 34.69 ns 25.71 ns 0.74 0.08 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Single>.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 25.713202773944737 < 32.51422787620426.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 51.27285340927814 (T) = (0 -25.545720689418086) / Math.Sqrt((1.2840223763310914 / (299)) + (0.2684515030987028 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24069750919258576 = (33.64366770646796 - 25.545720689418086) / 33.64366770646796 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractSerializer_ - Duration of single invocation 57.90 μs 46.08 μs 0.80 0.11 False
DataContractSerializer_BinaryXml_ - Duration of single invocation 35.42 μs 25.78 μs 0.73 0.05 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>.DataContractSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 46.078058931860035 < 54.87806467156262.
IsChangePoint: Marked as a change because one of 4/4/2023 8:16:00 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 49.83149283675738 (T) = (0 -44486.067895306325) / Math.Sqrt((7775364.319239066 / (299)) + (1209523.1847977063 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2787528039428056 = (61679.36338400491 - 44486.067895306325) / 61679.36338400491 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>.DataContractSerializer_BinaryXml_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 25.78166449728563 < 33.59866256278734.
IsChangePoint: Marked as a change because one of 4/4/2023 8:16:00 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 63.821109230050354 (T) = (0 -25948.035146251852) / Math.Sqrt((9258710.785482094 / (299)) + (86434.66978005363 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.322961299665756 = (38325.77832469797 - 25948.035146251852) / 38325.77832469797 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsKeyTrue<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableDictionary - Duration of single invocation 32.56 μs 23.28 μs 0.71 0.02 True
ImmutableSortedDictionary - Duration of single invocation 228.37 μs 201.56 μs 0.88 0.03 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyTrue<String, String>.ImmutableDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.276666817851957 < 31.195636302001144.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 71.22594015413127 (T) = (0 -23920.541027296473) / Math.Sqrt((613593.9474520386 / (299)) + (202112.10439563598 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.283162084386038 = (33369.525392373886 - 23920.541027296473) / 33369.525392373886 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsKeyTrue<String, String>.ImmutableSortedDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 201.5555608974359 < 236.43302482722356.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/17/2023 1:30:52 PM, 4/12/2023 4:47:47 AM, 5/16/2023 12:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 27.492926785744526 (T) = (0 -204560.2004836486) / Math.Sqrt((76053958.15734227 / (299)) + (7019637.410056181 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.08718744643561101 = (224098.80285374395 - 204560.2004836486) / 224098.80285374395 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_FromStream<ClassImplementingIXmlSerialiable>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractSerializer_ - Duration of single invocation 1.73 μs 1.37 μs 0.79 0.02 True
XmlSerializer_ - Duration of single invocation 2.39 μs 2.01 μs 0.84 0.01 True
DataContractSerializer_BinaryXml_ - Duration of single invocation 784.05 ns 520.37 ns 0.66 0.06 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;ClassImplementingIXmlSerialiable&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_FromStream<ClassImplementingIXmlSerialiable>.DataContractSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.3722480577703788 < 1.6516231257605178.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 43.2167655230964 (T) = (0 -1364.3877444600641) / Math.Sqrt((1201.1078647891816 / (299)) + (1031.741671303408 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2243459437459105 = (1759.0158053825953 - 1364.3877444600641) / 1759.0158053825953 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_FromStream<ClassImplementingIXmlSerialiable>.XmlSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.0092719768392957 < 2.275028410377417.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 33.17311491133883 (T) = (0 -2016.273075559938) / Math.Sqrt((512.978757214091 / (299)) + (1941.9907338276437 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16821978532096793 = (2424.0454869895875 - 2016.273075559938) / 2424.0454869895875 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Xml_FromStream<ClassImplementingIXmlSerialiable>.DataContractSerializer_BinaryXml_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 520.3743311266683 < 746.2217917717926.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 150.3671773810935 (T) = (0 -521.4111110645827) / Math.Sqrt((223.56853403323055 / (299)) + (33.0056784388664 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.34332111348536365 = (794.0122969873514 - 521.4111110645827) / 794.0122969873514 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_Matrix3x2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateRotationWithCenterBenchmark - Duration of single invocation 16.16 ns 14.60 ns 0.90 0.06 False
CreateRotationBenchmark - Duration of single invocation 16.12 ns 14.27 ns 0.89 0.04 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Matrix3x2.CreateRotationWithCenterBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.604377001933146 < 15.379992055415578.
IsChangePoint: Marked as a change because one of 3/17/2023 8:17:18 AM, 3/20/2023 8:31:51 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 20.58318163930625 (T) = (0 -14.467380423609832) / Math.Sqrt((2.5240597092549937 / (299)) + (0.032588542644997054 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1295775198277345 = (16.621101537665467 - 14.467380423609832) / 16.621101537665467 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.CreateRotationBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.271741179578015 < 15.329164274908528.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/25/2023 6:46:56 PM, 5/1/2023 3:42:23 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 30.620456560370048 (T) = (0 -13.998301746148714) / Math.Sqrt((0.3969101557226289 / (299)) + (0.05435626697077267 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13967641941250938 = (16.27097299435832 - 13.998301746148714) / 16.27097299435832 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Net.NetworkInformation.Tests.PhysicalAddressTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
PAMedium - Duration of single invocation 23.01 ns 20.26 ns 0.88 0.12 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.NetworkInformation.Tests.PhysicalAddressTests*'

Payloads

Baseline
Compare

Histogram

System.Net.NetworkInformation.Tests.PhysicalAddressTests.PAMedium

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.25883211783829 < 21.8191788670669.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 25.57021572107768 (T) = (0 -20.489809717007844) / Math.Sqrt((0.4006590643070723 / (299)) + (0.14610297532741823 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12277876558004126 = (23.357630792597302 - 20.489809717007844) / 23.357630792597302 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateInstance - Duration of single invocation 753.78 ns 673.67 ns 0.89 0.11 False
ServiceProvider - Duration of single invocation 22.80 ns 18.01 ns 0.79 0.19 False
Factory - Duration of single invocation 33.61 ns 20.06 ns 0.60 0.09 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 673.6711872858939 < 718.2143477199801.
IsChangePoint: Marked as a change because one of 4/11/2023 3:40:00 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 12.390709289002645 (T) = (0 -662.976839567167) / Math.Sqrt((606.1739396889157 / (299)) + (980.2357975875756 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14123072654169214 = (772.0081051541647 - 662.976839567167) / 772.0081051541647 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.ServiceProvider

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.0081805319751 < 22.429204433057375.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM, 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 47.47161339682725 (T) = (0 -17.54442885172449) / Math.Sqrt((2.7497044008968667 / (299)) + (0.19403193961892387 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.29589502018981817 = (24.917348058600943 - 17.54442885172449) / 24.917348058600943 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.Factory

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.060391367840204 < 32.92712810127192.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 80.45641454387139 (T) = (0 -20.260927940717654) / Math.Sqrt((2.4093321710310827 / (299)) + (0.26804879668201786 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4020797829189661 = (33.88567130181478 - 20.260927940717654) / 33.88567130181478 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
JsonNet_ - Duration of single invocation 39.56 μs 28.11 μs 0.71 0.03 True
SystemTextJson_Reflection_ - Duration of single invocation 15.50 μs 13.58 μs 0.88 0.07 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>.JsonNet_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 28.109010019440706 < 38.654016455550746.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 111.24518185509379 (T) = (0 -27973.231961813064) / Math.Sqrt((790504.7820365569 / (299)) + (97675.58289925886 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.28612256419752263 = (39184.92245152426 - 27973.231961813064) / 39184.92245152426 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>.SystemTextJson_Reflection_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.577701370671054 < 14.85800749310571.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 25.802975953536443 (T) = (0 -13465.918060343209) / Math.Sqrt((182380.2103716946 / (299)) + (94600.765507603 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.145424846544771 = (15757.441584742593 - 13465.918060343209) / 15757.441584742593 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_FromString<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
JsonNet_ - Duration of single invocation 3.03 μs 1.75 μs 0.58 0.05 True
SystemTextJson_Reflection_ - Duration of single invocation 1.41 μs 993.06 ns 0.71 0.06 True
SystemTextJson_SourceGen_ - Duration of single invocation 1.36 μs 1.01 μs 0.74 0.05 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_FromString<Location>.JsonNet_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.748554484562106 < 2.790205505591254.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 107.79204022833704 (T) = (0 -1768.5983007194066) / Math.Sqrt((5946.155458496482 / (299)) + (1446.5612911125047 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.41107408832275716 = (3003.0913322908364 - 1768.5983007194066) / 3003.0913322908364 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromString<Location>.SystemTextJson_Reflection_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 993.0595768786469 < 1.333278608757971.
IsChangePoint: Marked as a change because one of 4/25/2023 9:59:52 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 56.18328533741644 (T) = (0 -989.0553034410722) / Math.Sqrt((1521.2106580688421 / (299)) + (478.96602456323694 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26891864585452013 = (1352.8662683472808 - 989.0553034410722) / 1352.8662683472808 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_FromString<Location>.SystemTextJson_SourceGen_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0106942044771634 < 1.2872769449014327.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 87.30140088335776 (T) = (0 -1008.5298458957848) / Math.Sqrt((1036.0488902582858 / (299)) + (147.08164744192592 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24968777912865417 = (1344.146900239167 - 1008.5298458957848) / 1344.146900239167 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.Perf_Directory

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetCurrentDirectory - Duration of single invocation 73.78 ns 63.07 ns 0.85 0.12 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Directory*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Directory.GetCurrentDirectory

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 63.07371577918171 < 70.21812982271456.
IsChangePoint: Marked as a change because one of 3/24/2023 2:55:56 PM, 5/1/2023 6:56:14 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 8.56901701610848 (T) = (0 -63.868713304118465) / Math.Sqrt((9.300870504580065 / (299)) + (9.464399346558421 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1046622145055636 = (71.33476810525534 - 63.868713304118465) / 71.33476810525534 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IFooObjIsIFooInterAlia - Duration of single invocation 418.18 μs 390.54 μs 0.93 0.05 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf2.CastingPerf.IFooObjIsIFooInterAlia

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 390.54292067307694 < 397.00531507554945.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 4/12/2023 8:39:35 PM, 4/21/2023 8:59:32 PM, 5/16/2023 5:40:18 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 47.223809660688076 (T) = (0 -390158.07436648034) / Math.Sqrt((150036817.77802262 / (299)) + (263894.17698299215 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08042350902308604 = (424280.1748356954 - 390158.07436648034) / 424280.1748356954 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 17.34 ns 11.88 ns 0.69 0.06 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.879464389388449 < 16.100729538471438.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 209.75707074633507 (T) = (0 -11.899465510727195) / Math.Sqrt((0.1665877702902407 / (299)) + (0.0004102888660410541 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2995731086507116 = (16.9888758665509 - 11.899465510727195) / 16.9888758665509 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LoadXml - Duration of single invocation 2.57 μs 1.84 μs 0.72 0.02 True
GetDocumentElement - Duration of single invocation 6.63 ns 2.98 ns 0.45 0.09 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument*'

Payloads

Baseline
Compare

Histogram

XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument.LoadXml

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.842764491046183 < 2.4384047609552177.
IsChangePoint: Marked as a change because one of 4/25/2023 9:59:52 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 131.3558624481542 (T) = (0 -1838.7791673684546) / Math.Sqrt((3636.031681045964 / (299)) + (324.48352821506336 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.30325261497308925 = (2639.0901593372105 - 1838.7791673684546) / 2639.0901593372105 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument.GetDocumentElement

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.9787649710464428 < 6.29703878775107.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 122.02133568222462 (T) = (0 -2.9535540495172823) / Math.Sqrt((0.1073966234866044 / (299)) + (0.005594380144020697 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5372175068148306 = (6.38216460866747 - 2.9535540495172823) / 6.38216460866747 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SystemTextJson_Reflection_ - Duration of single invocation 768.11 ns 650.99 ns 0.85 0.05 False
JsonNet_ - Duration of single invocation 1.54 μs 893.06 ns 0.58 0.13 True
DataContractJsonSerializer_ - Duration of single invocation 2.44 μs 1.47 μs 0.60 0.18 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToStream<Location>.SystemTextJson_Reflection_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 650.9855271267492 < 729.6248143554474.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 44.68982259344271 (T) = (0 -644.7459756683573) / Math.Sqrt((288.723281560769 / (299)) + (117.73210384005739 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1799447293924977 = (786.2225861809599 - 644.7459756683573) / 786.2225861809599 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToStream<Location>.JsonNet_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 893.0612536528555 < 1.4622298981357096.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 39.92771552179285 (T) = (0 -917.5113557985694) / Math.Sqrt((1401.3793358169614 / (299)) + (3110.2756509110063 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.40464796041795575 = (1541.1240657589601 - 917.5113557985694) / 1541.1240657589601 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

MicroBenchmarks.Serializers.Json_ToStream<Location>.DataContractJsonSerializer_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.4718344187068941 < 2.320644107215748.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 43.61258894636518 (T) = (0 -1510.2919019986355) / Math.Sqrt((5574.565462900744 / (299)) + (5975.211667669417 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3870762810132911 = (2464.078082172222 - 1510.2919019986355) / 2464.078082172222 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Struct.SpanWrapper

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WrapperSum - Duration of single invocation 8.42 μs 6.99 μs 0.83 0.01 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Struct.SpanWrapper*'

Payloads

Baseline
Compare

Histogram

Struct.SpanWrapper.WrapperSum

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.985793149437771 < 8.001988903095622.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 669.2589023588317 (T) = (0 -6992.3294357942195) / Math.Sqrt((21.252464187909077 / (93)) + (56.51774029599286 / (13))) is greater than 1.9830375264834914 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (93) + (13) - 2, .975) and 0.16995008043934468 = (8423.986643472303 - 6992.3294357942195) / 8423.986643472303 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Net.Tests.Perf_WebUtility

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HtmlDecode_Entities - Duration of single invocation 147.40 ns 132.86 ns 0.90 0.04 False
Decode_DecodingRequired - Duration of single invocation 96.04 ns 76.27 ns 0.79 0.09 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Tests.Perf_WebUtility*'

Payloads

Baseline
Compare

Histogram

System.Net.Tests.Perf_WebUtility.HtmlDecode_Entities

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 132.8631095485744 < 140.11260230564068.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 5.967558418694336 (T) = (0 -137.76654700472346) / Math.Sqrt((6.177580014309317 / (299)) + (36.346195965154855 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06776928302341109 = (147.78160008664807 - 137.76654700472346) / 147.78160008664807 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Net.Tests.Perf_WebUtility.Decode_DecodingRequired

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 76.26807723397422 < 90.62217825384037.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/1/2023 10:17:01 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 15.90293963291589 (T) = (0 -75.8962262056263) / Math.Sqrt((93.87103239637246 / (299)) + (14.113475391876333 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19864749324958147 = (94.7101625892389 - 75.8962262056263) / 94.7101625892389 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Runtime.Serialization.Formatters.Tests.Perf_BinaryFormatter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeLargeList - Duration of single invocation 174.07 ms 142.64 ms 0.82 0.01 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Serialization.Formatters.Tests.Perf_BinaryFormatter*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Serialization.Formatters.Tests.Perf_BinaryFormatter.DeserializeLargeList

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 142.63964375 < 165.3492852083333.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 78.63972548405575 (T) = (0 -143603745.99496153) / Math.Sqrt((2805145539667.079 / (299)) + (1648403486068.5042 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16811272769463714 = (172624045.0788488 - 143603745.99496153) / 172624045.0788488 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Binary_FromStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinaryFormatter_ - Duration of single invocation 21.61 μs 16.60 μs 0.77 0.01 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_FromStream&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_FromStream<IndexViewModel>.BinaryFormatter_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.59552695342892 < 20.126846443250745.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 148.08509025719417 (T) = (0 -16582.199633068867) / Math.Sqrt((43243.513893248724 / (299)) + (10958.047858291742 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21914044023601228 = (21235.828422310387 - 16582.199633068867) / 21235.828422310387 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.Logging.EventSourceLogger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NestedScopes_TwoMessages - Duration of single invocation 2.01 μs 1.60 μs 0.80 0.03 True
NestedScopes_TwoMessages - Duration of single invocation 113.01 ns 95.87 ns 0.85 0.19 False
NestedScopes_TwoMessages - Duration of single invocation 988.27 ns 859.64 ns 0.87 0.04 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.EventSourceLogger*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: True, Json: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.603212645655711 < 1.9046703943726926.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 53.302323157383405 (T) = (0 -1607.676119847366) / Math.Sqrt((956.8540674074878 / (299)) + (754.8145802396139 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20603744798472973 = (2024.8765080502758 - 1607.676119847366) / 2024.8765080502758 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: False, Json: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 95.87002369336217 < 106.2719194070323.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/7/2023 9:12:04 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 20.700345702781494 (T) = (0 -97.01379462353142) / Math.Sqrt((43.5312584452793 / (299)) + (12.042072532314235 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1809410008665262 = (118.44542911581155 - 97.01379462353142) / 118.44542911581155 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: True, Json: False)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 859.6435269935145 < 915.303592928839.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 44.36909382077352 (T) = (0 -851.6352169676745) / Math.Sqrt((704.4144045328053 / (299)) + (105.24771187241704 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14415202791105422 = (995.0776828845095 - 851.6352169676745) / 995.0776828845095 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 65.21 ns 50.59 ns 0.78 0.08 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt16>.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 50.59413647028791 < 61.76854346441297.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/16/2023 8:14:39 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 33.205480140265514 (T) = (0 -48.860863407471896) / Math.Sqrt((6.218524874379836 / (299)) + (3.163099925617708 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2588506889356227 = (65.92580290913577 - 48.860863407471896) / 65.92580290913577 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tasks.ValueTaskPerfTest

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateAndAwait_FromDelayedTCS - Duration of single invocation 155.49 ns 123.41 ns 0.79 0.16 False
CreateAndAwait_FromCompletedValueTaskSource - Duration of single invocation 17.40 ns 14.03 ns 0.81 0.24 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tasks.ValueTaskPerfTest*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromDelayedTCS

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 123.405432837737 < 170.91206525778065.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/13/2023 1:49:08 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 57.87101973823367 (T) = (0 -121.45563512608237) / Math.Sqrt((170.15329072712788 / (299)) + (2.313677371061038 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.29169954133750325 = (171.4747373670072 - 121.45563512608237) / 171.4747373670072 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromCompletedValueTaskSource

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.025343341998816 < 19.655788361190588.
IsChangePoint: Marked as a change because one of 3/4/2023 1:07:20 AM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 31.92047628104934 (T) = (0 -14.715947972259812) / Math.Sqrt((3.085130013567121 / (299)) + (0.4915157495792361 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.32242724869973655 = (21.718624227464694 - 14.715947972259812) / 21.718624227464694 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Xml.Linq.Perf_XDocument

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 1.90 μs 1.46 μs 0.77 0.02 True
CreateWithRootlEement - Duration of single invocation 48.93 ns 40.71 ns 0.83 0.18 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XDocument*'

Payloads

Baseline
Compare

Histogram

System.Xml.Linq.Perf_XDocument.Parse

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.4649954942066983 < 1.80426159447246.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 68.60438456306323 (T) = (0 -1480.213186005907) / Math.Sqrt((3041.921717144781 / (299)) + (509.0515276761109 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24558417700665053 = (1962.0654033113456 - 1480.213186005907) / 1962.0654033113456 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Xml.Linq.Perf_XDocument.CreateWithRootlEement

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 40.70861926248843 < 48.880886854372086.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 21.696158182983353 (T) = (0 -40.27747936818469) / Math.Sqrt((3.40341324707421 / (299)) + (3.154119467991648 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2135171827552828 = (51.21215427094601 - 40.27747936818469) / 51.21215427094601 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.Permutate

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 618.84 ms 574.59 ms 0.93 0.08 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Permutate*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.Permutate.Test

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 574.5913583333333 < 588.038874142857.
IsChangePoint: Marked as a change because one of 5/11/2023 1:19:27 PM, 5/12/2023 4:02:48 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 15.781768118678135 (T) = (0 -575334404.3913778) / Math.Sqrt((3372627816070588 / (299)) + (598021567179.4569 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08451235429466386 = (628445842.0496895 - 575334404.3913778) / 628445842.0496895 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IterateFor<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IList - Duration of single invocation 2.15 μs 365.86 ns 0.17 0.19 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateFor<Int32>.IList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 365.85826311690147 < 1.971286513445962.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 457.9874564609097 (T) = (0 -365.97952338759353) / Math.Sqrt((3945.2717090453025 / (299)) + (0.014195901632140184 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.8196858507679234 = (2029.67723246473 - 365.97952338759353) / 2029.67723246473 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Concurrent.IsEmpty<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 4.42 ns 2.88 ns 0.65 0.03 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.IsEmpty&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.IsEmpty<String>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.884617764303892 < 4.186926003731965.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 118.457316177934 (T) = (0 -2.8723231556240614) / Math.Sqrt((0.04693315584829864 / (299)) + (8.310504009653601E-05 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3451682688329475 = (4.386353041421736 - 2.8723231556240614) / 4.386353041421736 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Stack - Duration of single invocation 904.01 μs 818.21 μs 0.91 0.03 False
ICollection - Duration of single invocation 899.50 μs 817.29 μs 0.91 0.03 False
ImmutableHashSet - Duration of single invocation 26.52 μs 23.03 μs 0.87 0.01 False
Array - Duration of single invocation 757.85 μs 392.52 μs 0.52 0.02 True

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<String>.Stack(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 818.2098541666667 < 854.2849153025792.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 17.531012733007184 (T) = (0 -821165.2392109619) / Math.Sqrt((1542053016.08224 / (299)) + (18474645.417535 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.05191433405114278 = (866129.7904858928 - 821165.2392109619) / 866129.7904858928 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsFalse<String>.ICollection(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 817.2943870192307 < 850.6724562665344.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 20.76800511495684 (T) = (0 -818644.1969443765) / Math.Sqrt((1309351427.5694697 / (299)) + (1206802.990236217 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.05091573309141306 = (862562.182819564 - 818644.1969443765) / 862562.182819564 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsFalse<String>.ImmutableHashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.02754563492064 < 25.36810579536752.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 31.783886281515286 (T) = (0 -23303.18921936897) / Math.Sqrt((184659.59403554394 / (299)) + (105719.32934243768 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11314710515373771 = (26276.273500137388 - 23303.18921936897) / 26276.273500137388 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsFalse<String>.Array(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 392.5199166666667 < 719.429324369331.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 173.41676247136255 (T) = (0 -392801.44756885746) / Math.Sqrt((879038986.2258301 / (299)) + (262845.1669424069 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4316832545783853 = (691166.4150903235 - 392801.44756885746) / 691166.4150903235 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IntegerFormatting - Duration of single invocation 1.25 ms 776.76 μs 0.62 0.05 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.IntegerFormatting

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 776.7600198412697 < 1.1835573858173076.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 14.07980463399017 (T) = (0 -833784.0264299948) / Math.Sqrt((3718714981.518872 / (299)) + (5130678103.761085 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.254131453428951 = (1117869.9386951174 - 833784.0264299948) / 1117869.9386951174 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 9.50 ns 6.89 ns 0.72 0.03 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.8864881343746145 < 9.048899872040385.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 104.02614823633392 (T) = (0 -6.955037088825427) / Math.Sqrt((0.032360758299359055 / (299)) + (0.006006843362126291 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26318058499810004 = (9.439269578431905 - 6.955037088825427) / 9.439269578431905 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IterateForNonGeneric<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ArrayList - Duration of single invocation 1.72 μs 366.15 ns 0.21 0.00 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForNonGeneric&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForNonGeneric<String>.ArrayList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 366.14728376694563 < 1.6348703292065159.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 15205.861470373444 (T) = (0 -366.296117346949) / Math.Sqrt((0.6690192813553927 / (299)) + (0.07408361277201198 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7871511075355678 = (1720.9209458684802 - 366.296117346949) / 1720.9209458684802 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Concurrent.Count<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue_EnqueueCountDequeue - Duration of single invocation 14.59 ns 13.32 ns 0.91 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.Count&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.Count<Int32>.Queue_EnqueueCountDequeue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.319160162014892 < 13.859593506569405.
IsChangePoint: Marked as a change because one of 3/25/2023 10:49:06 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 11.308776933403864 (T) = (0 -13.453666956133864) / Math.Sqrt((0.06770512609293537 / (299)) + (0.12791544832282278 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07777536446538691 = (14.588275391640108 - 13.453666956133864) / 14.588275391640108 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IndexerSet<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentDictionary - Duration of single invocation 22.40 μs 16.66 μs 0.74 0.01 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<String>.ConcurrentDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.656126044832828 < 21.342323429676174.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 155.91729698378242 (T) = (0 -16635.844211246396) / Math.Sqrt((135725.16328567566 / (299)) + (7555.7986675474485 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23168154328854243 = (21652.27721126319 - 16635.844211246396) / 21652.27721126319 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Binary_ToStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinaryFormatter_ - Duration of single invocation 923.55 μs 686.74 μs 0.74 0.04 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_ToStream&lt;CollectionsOfPrimitives&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_ToStream<CollectionsOfPrimitives>.BinaryFormatter_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 686.7424493554328 < 880.532512839682.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 190.62785806701336 (T) = (0 -684352.8152974858) / Math.Sqrt((309225139.3410536 / (299)) + (11667628.792955622 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27909638828230465 = (949298.6360088832 - 684352.8152974858) / 949298.6360088832 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.ConsoleTests.Perf_Console

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OpenStandardInput - Duration of single invocation 182.11 ns 172.01 ns 0.94 0.00 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.ConsoleTests.Perf_Console*'

Payloads

Baseline
Compare

Histogram

System.ConsoleTests.Perf_Console.OpenStandardInput

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 172.01028827161502 < 173.91627745534507.
IsChangePoint: Marked as a change because one of 3/25/2023 6:47:18 AM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 58.59338098972917 (T) = (0 -171.66943232595185) / Math.Sqrt((7.775603253513171 / (299)) + (0.42209560358755566 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07624230279499879 = (185.83816172289474 - 171.66943232595185) / 185.83816172289474 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 70.68 ns 45.39 ns 0.64 0.04 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 45.38831794731161 < 67.53847536905123.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 71.80116447557229 (T) = (0 -45.69628499850756) / Math.Sqrt((5.716645124489806 / (299)) + (1.0167348070997435 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.32894899229633434 = (68.09658949008973 - 45.69628499850756) / 68.09658949008973 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IterateForNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ArrayList - Duration of single invocation 1.72 μs 366.66 ns 0.21 0.00 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForNonGeneric&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForNonGeneric<Int32>.ArrayList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 366.6550519200937 < 1.634108818278277.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 15525.977818119 (T) = (0 -366.61669916308614) / Math.Sqrt((0.4251433722942496 / (299)) + (0.08036800333210752 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7869126529271837 = (1720.4996176418008 - 366.61669916308614) / 1720.4996176418008 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in V8.Richards.Support

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Bench - Duration of single invocation 95.56 μs 79.90 μs 0.84 0.01 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'V8.Richards.Support*'

Payloads

Baseline
Compare

Histogram

V8.Richards.Support.Bench

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 79.89827380952381 < 89.71628541528919.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 51.8691243233337 (T) = (0 -80665.81072529874) / Math.Sqrt((785631.8483848915 / (299)) + (919722.1147603244 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14834075307536704 = (94716.06281101907 - 80665.81072529874) / 94716.06281101907 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in JetStream.TimeSeriesSegmentation

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MaximizeSchwarzCriterion - Duration of single invocation 94.00 ms 54.00 ms 0.57 0.00 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'JetStream.TimeSeriesSegmentation*'

Payloads

Baseline
Compare

Histogram

JetStream.TimeSeriesSegmentation.MaximizeSchwarzCriterion

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 53.99865384615385 < 85.45289826923076.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 320.38458219069344 (T) = (0 -54064808.080445185) / Math.Sqrt((4041548748346.6694 / (299)) + (10006586953.294985 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.41462523464642237 = (92359307.71253695 - 54064808.080445185) / 92359307.71253695 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Binary_FromStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinaryFormatter_ - Duration of single invocation 4.64 μs 3.81 μs 0.82 0.03 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_FromStream&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_FromStream<Location>.BinaryFormatter_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.813090509292098 < 4.43341656102486.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 53.55193294222726 (T) = (0 -3802.568249559518) / Math.Sqrt((3252.5508886774605 / (299)) + (3479.520444720531 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.190307881850047 = (4696.313776955022 - 3802.568249559518) / 4696.313776955022 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Binary_ToStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinaryFormatter_ - Duration of single invocation 1.92 μs 1.44 μs 0.75 0.01 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_ToStream&lt;LoginViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_ToStream<LoginViewModel>.BinaryFormatter_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.444391025641026 < 1.8278361161763246.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 79.36326082224058 (T) = (0 -1470.7964793002152) / Math.Sqrt((515.5952121807442 / (299)) + (403.2155529371781 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23591453345973057 = (1924.910947409965 - 1470.7964793002152) / 1924.910947409965 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 17.78 ns 12.95 ns 0.73 0.00 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt64>.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.949297768185305 < 16.86686373389988.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 257.4120137977761 (T) = (0 -12.931775533838252) / Math.Sqrt((0.10171544810506002 / (299)) + (0.0007408176198374396 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2840254386295546 = (18.061780727356524 - 12.931775533838252) / 18.061780727356524 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchF.DMath

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 654.51 ms 494.49 ms 0.76 0.07 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.DMath*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchF.DMath.Test

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 494.4855266666667 < 627.35717625.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 176.09763075863535 (T) = (0 -494326575.5424064) / Math.Sqrt((246291944356686.8 / (299)) + (254203896599.6276 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2464957191455656 = (656036850.8880479 - 494326575.5424064) / 656036850.8880479 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Binary_FromStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinaryFormatter_ - Duration of single invocation 69.00 μs 51.57 μs 0.75 0.01 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_FromStream&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_FromStream<MyEventsListerViewModel>.BinaryFormatter_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 51.57116016780139 < 65.42451481144607.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 92.8771003720994 (T) = (0 -51532.0558217248) / Math.Sqrt((377525.3595772121 / (299)) + (411855.6104054407 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24649338809369392 = (68389.65313304045 - 51532.0558217248) / 68389.65313304045 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Reflection.Activator<EmptyStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateInstanceNames - Duration of single invocation 2.32 μs 2.14 μs 0.92 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Activator&lt;EmptyStruct&gt;*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Activator<EmptyStruct>.CreateInstanceNames

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.136606321626422 < 2.1849756468246997.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 33.630025550847456 (T) = (0 -2119.9078903051904) / Math.Sqrt((6874.926134091033 / (299)) + (654.8475460704869 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11962578464973142 = (2407.962265752816 - 2119.9078903051904) / 2407.962265752816 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Load - Duration of single invocation 106.94 μs 84.82 μs 0.79 0.03 True
Load - Duration of single invocation 48.47 μs 36.21 μs 0.75 0.04 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName: "repeated.xml")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 84.81977611940299 < 101.24333433118285.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 68.06920359635018 (T) = (0 -83280.11517906877) / Math.Sqrt((2936382.28515587 / (299)) + (1520764.910686609 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22543899830958106 = (107519.1172771627 - 83280.11517906877) / 107519.1172771627 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName: "simple.xml")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 36.21110306191442 < 46.145117827989644.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 30.44097823809248 (T) = (0 -37253.32369463092) / Math.Sqrt((850404.1456114227 / (299)) + (1974032.8248907577 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.243219595988892 = (49226.06808683185 - 37253.32369463092) / 49226.06808683185 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 26, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.DictionarySequentialKeys

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryGetValue_17_Int_32ByteValue - Duration of single invocation 4.75 ns 2.39 ns 0.50 0.01 True
TryGetValue_17_Int_32ByteRefsValue - Duration of single invocation 4.71 ns 2.46 ns 0.52 0.02 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.DictionarySequentialKeys*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_32ByteValue

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.3898866267040537 < 4.5104014650901165.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 253.30425243606896 (T) = (0 -2.452799624623374) / Math.Sqrt((0.004225284744222498 / (299)) + (0.0008635401163835079 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.48103208375881934 = (4.726303009998563 - 2.452799624623374) / 4.726303009998563 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_32ByteRefsValue

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.4640920274686136 < 4.481153300442759.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 186.9145906729396 (T) = (0 -2.4300650454498802) / Math.Sqrt((0.0036603435340256704 / (299)) + (0.0017913699276581412 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.48511146076268413 = (4.719594359294615 - 2.4300650454498802) / 4.719594359294615 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 17.95 ns 13.83 ns 0.77 0.03 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.834867008751397 < 17.08268178452467.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 211.30083121566804 (T) = (0 -13.8400768055773) / Math.Sqrt((0.10549637134467695 / (299)) + (0.0006903464958598113 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23524188350816183 = (18.097325817299787 - 13.8400768055773) / 18.097325817299787 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.Logging.Formatting

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NoArguments - Duration of single invocation 34.99 ns 28.78 ns 0.82 0.06 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.Formatting*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.Formatting.NoArguments

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 28.775519608536857 < 33.06548011529844.
IsChangePoint: Marked as a change because one of 3/3/2023 11:00:07 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 43.558517691454284 (T) = (0 -28.29148499343477) / Math.Sqrt((1.8608159687482122 / (299)) + (0.18822155377601477 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18135160243981882 = (34.558774044818165 - 28.29148499343477) / 34.558774044818165 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 132.36 ns 100.49 ns 0.76 0.06 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<SByte>.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 100.49116156989423 < 125.80324363798353.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 59.20480470803139 (T) = (0 -100.881419524202) / Math.Sqrt((20.592669588274333 / (299)) + (4.107919231751063 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.266906738539286 = (137.61062176890323 - 100.881419524202) / 137.61062176890323 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 30.26 ns 25.13 ns 0.83 0.12 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.GetHashCodeBenchmark

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 25.13343666033209 < 29.80989251097874.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 4/26/2023 4:28:25 PM, 5/1/2023 3:42:23 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 21.334411411769473 (T) = (0 -24.863916381266854) / Math.Sqrt((2.5118044657910255 / (299)) + (1.024770308837079 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20218324023796877 = (31.164946182232544 - 24.863916381266854) / 31.164946182232544 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Xml.Linq.Perf_XElementList

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Enumerator - Duration of single invocation 244.13 ns 194.22 ns 0.80 0.20 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XElementList*'

Payloads

Baseline
Compare

Histogram

System.Xml.Linq.Perf_XElementList.Enumerator

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 194.22313319400558 < 235.7359684579467.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 35.654851925543966 (T) = (0 -197.38458025640864) / Math.Sqrt((126.53894896284872 / (299)) + (14.38821837539295 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1826288801290909 = (241.48709864814214 - 197.38458025640864) / 241.48709864814214 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Binary_ToStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinaryFormatter_ - Duration of single invocation 3.56 μs 2.63 μs 0.74 0.02 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_ToStream&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_ToStream<Location>.BinaryFormatter_

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.6273521002726374 < 3.3798591644187534.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 74.20964058787408 (T) = (0 -2653.114616749167) / Math.Sqrt((3092.3426536478532 / (299)) + (1603.9175385158235 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2443975122283374 = (3511.25712221175 - 2653.114616749167) / 3511.25712221175 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.MDBenchI.MDPuzzle

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 431.90 ms 372.18 ms 0.86 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDPuzzle*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDPuzzle.Test

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 372.18338 < 410.5930716666666.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 27.395641374291053 (T) = (0 -363889207.97407717) / Math.Sqrt((125091880817578.14 / (299)) + (115966852591557.8 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18703815058776507 = (447609206.0619649 - 363889207.97407717) / 447609206.0619649 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Doubles

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteDoubles - Duration of single invocation 20.95 ms 17.89 ms 0.85 0.08 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Doubles*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: True, SkipValidation: False)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.89363833333333 < 19.930821875.
IsChangePoint: Marked as a change because one of 3/17/2023 5:15:23 PM, 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 15.039287973856908 (T) = (0 -18833336.370808676) / Math.Sqrt((1043805992415.8896 / (299)) + (432912516522.79083 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13282589007582699 = (21718056.56473703 - 18833336.370808676) / 21718056.56473703 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in SciMark2.kernel

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
benchMonteCarlo - Duration of single invocation 775.47 ms 632.98 ms 0.82 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'SciMark2.kernel*'

Payloads

Baseline
Compare

Histogram

SciMark2.kernel.benchMonteCarlo

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 632.9763615384616 < 736.2278039285713.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/12/2023 8:39:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM falls between 5/13/2023 2:42:33 PM and 5/22/2023 7:40:07 AM.
IsImprovementStdDev: Marked as improvement because 83.80030490807927 (T) = (0 -636374907.6444068) / Math.Sqrt((429381449734760.75 / (299)) + (4678218504521.978 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15000113668362405 = (748677363.122006 - 636374907.6444068) / 748677363.122006 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@kunalspathak
Copy link
Member

dotnet/runtime#86225

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@EgorBo EgorBo

Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-improvement runtime-coreclr untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@EgorBo @kunalspathak