Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 271 Improvements on 12/18/2022 12:43:31 PM #10989

Closed
performanceautofiler bot opened this issue Dec 20, 2022 · 12 comments
Closed

[Perf] Linux/x64: 271 Improvements on 12/18/2022 12:43:31 PM #10989

performanceautofiler bot opened this issue Dec 20, 2022 · 12 comments
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-improvement runtime-mono untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Dec 20, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ArrayList>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToUtf8Bytes - Duration of single invocation 147.56 μs 139.61 μs 0.95 0.03 False
SerializeToWriter - Duration of single invocation 145.13 μs 136.58 μs 0.94 0.04 False
SerializeObjectProperty - Duration of single invocation 162.56 μs 151.55 μs 0.93 0.03 False
SerializeToString - Duration of single invocation 157.19 μs 146.31 μs 0.93 0.03 False
SerializeToStream - Duration of single invocation 154.53 μs 143.35 μs 0.93 0.02 False
SerializeToWriter - Duration of single invocation 146.67 μs 134.77 μs 0.92 0.03 False
SerializeObjectProperty - Duration of single invocation 165.59 μs 149.49 μs 0.90 0.07 False
SerializeToUtf8Bytes - Duration of single invocation 147.65 μs 134.55 μs 0.91 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ArrayList&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToUtf8Bytes(Mode: Reflection)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 139.60520013020832 < 139.88632540383134.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 19.582591843446306 (T) = (0 -137074.0503684021) / Math.Sqrt((11260451.626662632 / (112)) + (2104682.6791992784 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.09351725457206608 = (151215.28904964644 - 137074.0503684021) / 151215.28904964644 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ArrayList&gt;.SerializeToWriter(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 136.58362028985505 < 140.8617335514937.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 17.563696068531524 (T) = (0 -135304.0153602907) / Math.Sqrt((14806426.177802792 / (112)) + (3522080.4928698335 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.10613100370212532 = (151368.95442249093 - 135304.0153602907) / 151368.95442249093 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ArrayList&gt;.SerializeObjectProperty(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 151.5509594150641 < 155.9722488409133.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 25.531042487982283 (T) = (0 -152840.70298229821) / Math.Sqrt((13550015.216580853 / (111)) + (1217932.2743891713 / (5))) is greater than 1.9809922979755963 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (111) + (5) - 2, .975) and 0.09174362781119613 = (168279.2520507926 - 152840.70298229821) / 168279.2520507926 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ArrayList&gt;.SerializeToString(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 146.30764102891155 < 150.1437761006107.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 9.734595981580515 (T) = (0 -147187.2012415023) / Math.Sqrt((12755344.437620495 / (112)) + (9513857.351707894 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.08585745012993322 = (161011.2134725848 - 147187.2012415023) / 161011.2134725848 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ArrayList&gt;.SerializeToStream(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 143.34711741354974 < 146.32970901339948.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 21.55948376298271 (T) = (0 -142425.0098692413) / Math.Sqrt((10406519.66030258 / (112)) + (1804713.1650140192 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.09254195033964903 = (156949.41482148846 - 142425.0098692413) / 156949.41482148846 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ArrayList&gt;.SerializeToWriter(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 134.76530138888893 < 139.3113243916947.
IsChangePoint: Marked as a change because one of 12/17/2022 11:39:38 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 21.37301322690889 (T) = (0 -135943.632081513) / Math.Sqrt((14890951.555579785 / (110)) + (2282854.0706597003 / (7))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (110) + (7) - 2, .975) and 0.09649817278237652 = (150463.04056756347 - 135943.632081513) / 150463.04056756347 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ArrayList&gt;.SerializeObjectProperty(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 149.4912813888889 < 157.60412507738096.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 12.37143384168435 (T) = (0 -151563.1367234743) / Math.Sqrt((16911583.51218432 / (112)) + (7813538.247704979 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.0965391933591155 = (167758.3970543162 - 151563.1367234743) / 167758.3970543162 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ArrayList&gt;.SerializeToUtf8Bytes(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 134.5519469827586 < 142.11988353075398.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 15.414759062321094 (T) = (0 -136448.113484448) / Math.Sqrt((12917040.548298292 / (112)) + (4780703.683078652 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.10469598785853156 = (152404.22430150755 - 136448.113484448) / 152404.22430150755 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
InequalityOperatorBenchmark - Duration of single invocation 27.99 ns 25.90 ns 0.93 0.21 False
GreaterThanAllBenchmark - Duration of single invocation 63.60 ns 57.70 ns 0.91 0.14 False
EqualityOperatorBenchmark - Duration of single invocation 27.71 ns 25.79 ns 0.93 0.20 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.InequalityOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.90373298571601 < 26.56188139279904.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 17.35165099212401 (T) = (0 -25.425800807465446) / Math.Sqrt((1.8248845901521966 / (166)) + (0.2169906001778006 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.1373056037188248 = (29.472546613341507 - 25.425800807465446) / 29.472546613341507 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.GreaterThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 57.70311371807243 < 59.59533093731254.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 22.59512935143215 (T) = (0 -58.12558882273978) / Math.Sqrt((5.979795735830928 / (166)) + (0.4393010255742269 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.12035420752120912 = (66.07840260219427 - 58.12558882273978) / 66.07840260219427 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.789182224430633 < 26.28147168001466.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 13.821654875234065 (T) = (0 -25.395137080960886) / Math.Sqrt((1.22766341817982 / (166)) + (0.3571463196682807 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.13255151296480785 = (29.275671651417166 - 25.395137080960886) / 29.275671651417166 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeObjectProperty - Duration of single invocation 297.94 μs 267.93 μs 0.90 0.04 False
SerializeToUtf8Bytes - Duration of single invocation 271.39 μs 240.49 μs 0.89 0.03 False
SerializeToWriter - Duration of single invocation 274.00 μs 242.55 μs 0.89 0.03 False
SerializeObjectProperty - Duration of single invocation 345.68 μs 307.60 μs 0.89 0.03 False
SerializeToWriter - Duration of single invocation 310.68 μs 290.13 μs 0.93 0.03 False
SerializeToString - Duration of single invocation 342.79 μs 308.25 μs 0.90 0.03 False
SerializeToString - Duration of single invocation 288.13 μs 259.31 μs 0.90 0.03 False
SerializeToUtf8Bytes - Duration of single invocation 323.12 μs 284.13 μs 0.88 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeObjectProperty(Mode: SourceGen)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 267.92779812885806 < 284.17986221796383.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 26.6411037123269 (T) = (0 -271837.97035340214) / Math.Sqrt((39202996.06417416 / (111)) + (6156870.111081981 / (5))) is greater than 1.9809922979755963 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (111) + (5) - 2, .975) and 0.10981820652001666 = (305373.5454313296 - 271837.97035340214) / 305373.5454313296 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;.SerializeToUtf8Bytes(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 240.48659798076923 < 256.722414996408.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 13.42087696600117 (T) = (0 -243232.93084391532) / Math.Sqrt((36391894.927262515 / (112)) + (28591532.078371894 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.1194405783363009 = (276225.4594747953 - 243232.93084391532) / 276225.4594747953 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;.SerializeToWriter(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 242.54669562146893 < 253.9492632039447.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 15.238520708030471 (T) = (0 -244211.61726614516) / Math.Sqrt((35215258.53965385 / (112)) + (17412734.960411813 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.10840796507255175 = (273905.11321247666 - 244211.61726614516) / 273905.11321247666 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;.SerializeObjectProperty(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 307.5983620915033 < 328.80959142100227.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 10.19407262806055 (T) = (0 -317773.08933266107) / Math.Sqrt((36104340.96156717 / (112)) + (52402923.66067074 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.0953820970727645 = (351278.7977160139 - 317773.08933266107) / 351278.7977160139 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;.SerializeToWriter(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 290.12691266233765 < 300.04615008705355.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 18.398976364446565 (T) = (0 -285310.338343866) / Math.Sqrt((39980775.56376865 / (112)) + (17730017.48639463 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.11300443068316635 = (321659.2598806472 - 285310.338343866) / 321659.2598806472 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;.SerializeToString(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 308.24632309173666 < 319.29211958994705.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 16.632375081086032 (T) = (0 -310955.59324739856) / Math.Sqrt((40499754.296211265 / (112)) + (15565415.59211646 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.0906646298420101 = (341959.1972908438 - 310955.59324739856) / 341959.1972908438 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;.SerializeToString(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 259.31193060897436 < 273.72376082717795.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 38.173579071811176 (T) = (0 -260554.45600488404) / Math.Sqrt((29573174.18134105 / (112)) + (2665100.715900038 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.11567123258695394 = (294635.28226848476 - 260554.45600488404) / 294635.28226848476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;.SerializeToUtf8Bytes(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 284.12781079545454 < 300.5225015332054.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 11.340893433300483 (T) = (0 -287811.6920717522) / Math.Sqrt((42140242.501750596 / (112)) + (39513962.41201585 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.1018324746021424 = (320443.21792224835 - 287811.6920717522) / 320443.21792224835 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 20, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Node.Tests.Perf_ParseThenWrite

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseThenWrite - Duration of single invocation 837.42 μs 141.71 μs 0.17 0.01 False
ParseThenWrite - Duration of single invocation 132.44 μs 29.25 μs 0.22 0.01 False
ParseThenWrite - Duration of single invocation 24.89 μs 11.75 μs 0.47 0.01 False
ParseThenWrite - Duration of single invocation 682.06 μs 207.28 μs 0.30 0.01 False
ParseThenWrite - Duration of single invocation 244.55 μs 65.72 μs 0.27 0.02 False
ParseThenWrite - Duration of single invocation 23.31 μs 11.02 μs 0.47 0.01 False
ParseThenWrite - Duration of single invocation 142.36 μs 34.00 μs 0.24 0.01 False
ParseThenWrite - Duration of single invocation 904.20 μs 173.44 μs 0.19 0.01 False
ParseThenWrite - Duration of single invocation 77.20 ms 12.09 ms 0.16 0.01 False
ParseThenWrite - Duration of single invocation 84.79 ms 15.48 ms 0.18 0.02 False
ParseThenWrite - Duration of single invocation 1.03 ms 222.10 μs 0.21 0.01 False
ParseThenWrite - Duration of single invocation 133.48 μs 37.22 μs 0.28 0.01 False
ParseThenWrite - Duration of single invocation 1.17 ms 282.76 μs 0.24 0.01 False
ParseThenWrite - Duration of single invocation 147.53 μs 42.09 μs 0.29 0.02 False
ParseThenWrite - Duration of single invocation 221.45 μs 54.70 μs 0.25 0.02 False
ParseThenWrite - Duration of single invocation 604.86 μs 154.26 μs 0.26 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Node.Tests.Perf_ParseThenWrite*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json4KB)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 141.7063715909091 < 791.9635073586309.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 460.41435095590845 (T) = (0 -140304.46080739005) / Math.Sqrt((360791076.2951089 / (166)) + (618974.8191293491 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.8325997270339007 = (838137.5867636935 - 140304.46080739005) / 838137.5867636935 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.253428794545854 < 125.86846362836711.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 515.0220634240277 (T) = (0 -29395.990884795872) / Math.Sqrt((4681956.209955351 / (166)) + (54750.1615305467 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.7761261940748974 = (131306.0755961345 - 29395.990884795872) / 131306.0755961345 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: HelloWorld)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.752532042376997 < 23.71481195918353.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 214.3404234594635 (T) = (0 -11827.265041410612) / Math.Sqrt((108245.04317759407 / (166)) + (15609.241089531031 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.5268088741932659 = (24994.68903024448 - 11827.265041410612) / 24994.68903024448 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 207.28130464285715 < 652.2388927859731.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 389.27585282137784 (T) = (0 -209953.03711148896) / Math.Sqrt((178549710.1816464 / (166)) + (2453613.125199992 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.6988378384002145 = (697142.8150077354 - 209953.03711148896) / 697142.8150077354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 65.7193517628205 < 229.87202071113782.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 356.5428279923249 (T) = (0 -65010.25635631239) / Math.Sqrt((21365237.58874781 / (166)) + (566989.7459537492 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.7296240070094545 = (240443.89310328188 - 65010.25635631239) / 240443.89310328188 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: HelloWorld)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.017125181095407 < 22.23781188403413.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 343.7189668496438 (T) = (0 -10922.982629669954) / Math.Sqrt((105932.27789367094 / (166)) + (3646.370171149294 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.5378128249288128 = (23633.244751950482 - 10922.982629669954) / 23633.244751950482 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json400B)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.99799611080786 < 134.85695164962124.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 423.97958367588006 (T) = (0 -33972.369241184584) / Math.Sqrt((4907166.926352361 / (166)) + (166042.63801316745 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.7576790817989626 = (140195.77630107853 - 33972.369241184584) / 140195.77630107853 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json4KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 173.44015680555555 < 857.394356994302.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 426.4160362480888 (T) = (0 -176434.77436534528) / Math.Sqrt((334262529.06345075 / (166)) + (4810906.274379482 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.8065462095753351 = (912025.4194970286 - 176434.77436534528) / 912025.4194970286 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.085266633699637 < 73.45305029687499.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 391.6430483923545 (T) = (0 -12430075.127025643) / Math.Sqrt((3266627937327.0947 / (165)) + (44075079104.56656 / (5))) is greater than 1.9741851911431834 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (165) + (5) - 2, .975) and 0.8420127265568772 = (78677698.88123682 - 12430075.127025643) / 78677698.88123682 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.476941958333335 < 80.12856605.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 433.77408485147964 (T) = (0 -15510851.731666666) / Math.Sqrt((3735216277430.699 / (166)) + (19359156487.46197 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.8195466831583509 = (85954927.30830607 - 15510851.731666666) / 85954927.30830607 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 222.09979413145544 < 0.9855704406423611.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 387.8931711165144 (T) = (0 -221003.58636401244) / Math.Sqrt((572263410.9262632 / (164)) + (5995886.187677167 / (5))) is greater than 1.974270957027852 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (164) + (5) - 2, .975) and 0.7916857548998898 = (1060914.4192602097 - 221003.58636401244) / 1060914.4192602097 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 37.22261852633279 < 128.7808640035477.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 372.7214376918519 (T) = (0 -37102.31637983877) / Math.Sqrt((10915315.928469494 / (166)) + (38756.261994826426 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.7314435551469607 = (138154.63039861908 - 37102.31637983877) / 138154.63039861908 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: BroadTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 282.75599340909093 < 1.1243235885262577.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 333.67569921922797 (T) = (0 -288311.34747158305) / Math.Sqrt((578302495.3446633 / (166)) + (20128294.684493907 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.7602784989313571 = (1202692.9006632022 - 288311.34747158305) / 1202692.9006632022 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 42.08547057502246 < 139.70082579946006.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 333.7125612410373 (T) = (0 -42448.97961910447) / Math.Sqrt((10235358.935498858 / (166)) + (204301.7190805016 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.7156782617766448 = (149299.09997158835 - 42448.97961910447) / 149299.09997158835 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfNumbers)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 54.70363259109312 < 210.82557406929558.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 389.681271700198 (T) = (0 -53925.48889800969) / Math.Sqrt((20675434.41297045 / (166)) + (280346.2311943914 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.7543680494015101 = (219537.76276505788 - 53925.48889800969) / 219537.76276505788 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: DeepTree)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 154.2567978145425 < 571.4262259268676.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 436.53160075830914 (T) = (0 -153583.48272856424) / Math.Sqrt((156763058.4935447 / (166)) + (839203.1858312851 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.7498448311013008 = (613952.8653543759 - 153583.48272856424) / 613952.8653543759 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 4.05 ms 3.76 ms 0.93 0.08 False
Count - Duration of single invocation 2.21 ms 1.98 ms 0.89 0.02 False
Count - Duration of single invocation 2.19 ms 1.94 ms 0.89 0.04 False
Count - Duration of single invocation 1.95 ms 1.70 ms 0.87 0.04 False
Count - Duration of single invocation 1.96 ms 1.71 ms 0.88 0.04 False
Count - Duration of single invocation 4.08 ms 3.61 ms 0.88 0.10 False
Count - Duration of single invocation 2.63 ms 2.37 ms 0.90 0.03 False
Count - Duration of single invocation 2.14 ms 1.88 ms 0.88 0.05 False
Count - Duration of single invocation 4.04 ms 3.63 ms 0.90 0.09 False
Count - Duration of single invocation 2.28 ms 2.04 ms 0.90 0.06 False
Count - Duration of single invocation 1.76 ms 1.53 ms 0.87 0.06 False
Count - Duration of single invocation 1.94 ms 1.53 ms 0.79 0.06 False
Count - Duration of single invocation 2.30 ms 2.11 ms 0.92 0.06 False
Count - Duration of single invocation 2.04 ms 1.76 ms 0.86 0.04 False
Count - Duration of single invocation 4.07 ms 3.65 ms 0.90 0.06 False
Count - Duration of single invocation 2.26 ms 2.05 ms 0.90 0.03 False
Count - Duration of single invocation 1.98 ms 1.73 ms 0.88 0.02 False
Count - Duration of single invocation 2.80 ms 2.50 ms 0.89 0.02 False
Count - Duration of single invocation 2.94 ms 2.62 ms 0.89 0.02 False
Count - Duration of single invocation 1.98 ms 1.74 ms 0.88 0.02 False
Count - Duration of single invocation 1.76 ms 1.53 ms 0.87 0.05 False
Count - Duration of single invocation 1.92 ms 1.68 ms 0.87 0.06 False
Count - Duration of single invocation 1.92 ms 1.67 ms 0.87 0.08 False
Count - Duration of single invocation 2.14 ms 1.77 ms 0.83 0.05 False
Count - Duration of single invocation 4.08 ms 3.70 ms 0.91 0.08 False
Count - Duration of single invocation 2.53 ms 2.23 ms 0.88 0.03 False
Count - Duration of single invocation 1.95 ms 1.67 ms 0.86 0.04 False
Count - Duration of single invocation 4.11 ms 3.59 ms 0.88 0.08 False
Count - Duration of single invocation 2.28 ms 2.02 ms 0.89 0.06 False
Count - Duration of single invocation 2.31 ms 2.05 ms 0.89 0.02 False
Count - Duration of single invocation 2.00 ms 1.76 ms 0.88 0.05 False
Count - Duration of single invocation 10.88 ms 10.07 ms 0.93 0.03 False
Count - Duration of single invocation 1.92 ms 1.67 ms 0.87 0.05 False
Count - Duration of single invocation 2.53 ms 2.24 ms 0.89 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: None)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.760821481601731 < 3.8725469705143216.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 12/2/2022 10:04:21 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 14.0724513684087 (T) = (0 -3657514.485216542) / Math.Sqrt((14502706475.209488 / (299)) + (4409156606.899628 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1050296920142524 = (4086743.9428781453 - 3657514.485216542) / 4086743.9428781453 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9770863502645497 < 2.1179075543516164.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 50.193155748384626 (T) = (0 -1971942.8639289415) / Math.Sqrt((1085808875.2622788 / (299)) + (156703360.3653179 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1308322686022741 = (2268771.369086403 - 1971942.8639289415) / 2268771.369086403 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.9380922818383164 < 2.0889649241391943.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 73.68143437408293 (T) = (0 -1945782.6904569133) / Math.Sqrt((2941700745.6930585 / (299)) + (36769109.53472376 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.13570410883362566 = (2251292.3066556104 - 1945782.6904569133) / 2251292.3066556104 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7040956694063927 < 1.8579097797712052.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 36.35139398683874 (T) = (0 -1723955.740988011) / Math.Sqrt((2010248525.7076297 / (299)) + (286654246.19892573 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1443920255731921 = (2014889.7538536033 - 1723955.740988011) / 2014889.7538536033 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7146594404597701 < 1.858208453250558.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 86.89100039368851 (T) = (0 -1718654.9695684523) / Math.Sqrt((2545415796.6524725 / (299)) + (15354337.252929792 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14681138791052925 = (2014390.423425182 - 1718654.9695684523) / 2014390.423425182 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.6084130724946695 < 3.8818092806669453.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/16/2022 11:42:33 AM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 25.879254579620085 (T) = (0 -3630515.330555481) / Math.Sqrt((14814697467.638271 / (299)) + (1271234515.4604664 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.11051290872740409 = (4081582.932655352 - 3630515.330555481) / 4081582.932655352 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.367268426415094 < 2.478154524234861.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 34.54435136660273 (T) = (0 -2357050.2852281122) / Math.Sqrt((2541431368.5778975 / (299)) + (401481054.228849 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12134533114871487 = (2682567.302930988 - 2357050.2852281122) / 2682567.302930988 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\\s+Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.8837984567099566 < 2.0329948434757834.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 36.42744697881732 (T) = (0 -1902302.7263285122) / Math.Sqrt((5081375606.313812 / (290)) + (272069656.4154582 / (5))) is greater than 1.9680934526838334 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (290) + (5) - 2, .975) and 0.13972089764688628 = (2211262.2765392773 - 1902302.7263285122) / 2211262.2765392773 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.625693024509804 < 3.8695417745619305.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 15.718731840670717 (T) = (0 -3659417.690002163) / Math.Sqrt((14723899072.385712 / (299)) + (3320268020.734618 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.10291409931604205 = (4079227.7386280876 - 3659417.690002163) / 4079227.7386280876 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.042108837533157 < 2.1689897065476194.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 55.18385257377758 (T) = (0 -2053340.7436852367) / Math.Sqrt((4916195558.325835 / (299)) + (59730056.73742889 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1252564721900755 = (2347363.173782079 - 2053340.7436852367) / 2347363.173782079 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5335371745192308 < 1.6693078794994214.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 47.664538587402504 (T) = (0 -1538061.3828074005) / Math.Sqrt((1286935989.255425 / (299)) + (148404910.53728354 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1530173321237976 = (1815930.173239635 - 1538061.3828074005) / 1815930.173239635 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.530948632711039 < 1.6727227853400999.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 59.33388466883093 (T) = (0 -1531343.949607892) / Math.Sqrt((6322370034.51816 / (299)) + (12809827.234727537 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.15871304836955655 = (1820239.7489228782 - 1531343.949607892) / 1820239.7489228782 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.114278964769648 < 2.188111486877957.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 15.971948534823856 (T) = (0 -2046670.3383698687) / Math.Sqrt((6406354979.5874815 / (299)) + (1659948384.2075803 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12793817412992406 = (2346932.611489855 - 2046670.3383698687) / 2346932.611489855 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7621131224586288 < 1.940345864590164.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 59.106876731311175 (T) = (0 -1780907.5481718022) / Math.Sqrt((1160488561.6463325 / (299)) + (114331792.78641307 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.146501403871044 = (2086596.9273401394 - 1780907.5481718022) / 2086596.9273401394 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.6515875052521016 < 3.854749481704345.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 21.893069795462868 (T) = (0 -3635700.415685486) / Math.Sqrt((11959211192.378506 / (299)) + (1834942824.0769587 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1083219219988656 = (4077368.845756081 - 3635700.415685486) / 4077368.845756081 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.0476224384823847 < 2.150715262594.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 52.652201132901894 (T) = (0 -2033649.2718669802) / Math.Sqrt((3545716624.311347 / (299)) + (105188263.83007629 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12929575006884245 = (2335637.240805671 - 2033649.2718669802) / 2335637.240805671 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\\s+Holmes", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.731601926340326 < 1.8780238510621277.
IsChangePoint: Marked as a change because one of 11/16/2022 11:42:33 AM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 101.25857352188233 (T) = (0 -1734439.858073412) / Math.Sqrt((1711855331.4931555 / (290)) + (13224397.9784748 / (5))) is greater than 1.9680934526838334 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (290) + (5) - 2, .975) and 0.14580072150740828 = (2030486.2129293573 - 1734439.858073412) / 2030486.2129293573 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.495739444666667 < 2.654673618506944.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 42.26335521459806 (T) = (0 -2514798.9151740745) / Math.Sqrt((2100698600.9979722 / (299)) + (233806012.30210608 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.10972899400098576 = (2824756.617061905 - 2514798.9151740745) / 2824756.617061905 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.6232067929078013 < 2.774005670733242.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 34.935225641158965 (T) = (0 -2628813.6730908095) / Math.Sqrt((19170790398.009457 / (299)) + (322176021.15992594 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1309452007203756 = (3024911.2889887746 - 2628813.6730908095) / 3024911.2889887746 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\\s+Holmes", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7434274755244752 < 1.878856583358731.
IsChangePoint: Marked as a change because one of 11/16/2022 11:42:33 AM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 101.07766016070917 (T) = (0 -1736539.1149994452) / Math.Sqrt((1072021271.7479738 / (290)) + (23141927.15453873 / (5))) is greater than 1.9680934526838334 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (290) + (5) - 2, .975) and 0.1437941437088377 = (2028179.4410069017 - 1736539.1149994452) / 2028179.4410069017 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5310536223776225 < 1.6701948955862713.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 61.595434192549966 (T) = (0 -1531890.0190228522) / Math.Sqrt((6432389843.374916 / (299)) + (2246366.6442145114 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.15855667937519768 = (1820550.4535770372 - 1531890.0190228522) / 1820550.4535770372 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.677127265926641 < 1.8250979269869172.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 47.13734297524239 (T) = (0 -1677916.816143976) / Math.Sqrt((14017846811.283194 / (299)) + (16587711.853870107 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.16600139537991537 = (2011894.0329742224 - 1677916.816143976) / 2011894.0329742224 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.674245253595398 < 1.8233868413528311.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 34.933359680039985 (T) = (0 -1684658.8757974126) / Math.Sqrt((5155420589.301477 / (299)) + (269059599.7969802 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14878592173763283 = (1979124.7804975274 - 1684658.8757974126) / 1979124.7804975274 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.773731444782168 < 1.934800447277517.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 40.15275208293732 (T) = (0 -1791482.555036862) / Math.Sqrt((1552674651.3220832 / (299)) + (252108807.39147615 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14320954658285695 = (2090922.6379587683 - 1791482.555036862) / 2090922.6379587683 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.70086919509804 < 3.8654428511446723.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/16/2022 11:42:33 AM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 21.989155111929108 (T) = (0 -3655834.6458355277) / Math.Sqrt((16164148960.17197 / (299)) + (1753492694.2922106 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.10794710290848834 = (4098226.3022239725 - 3655834.6458355277) / 4098226.3022239725 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.234510765943878 < 2.4012839340632515.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 44.35646260328857 (T) = (0 -2241272.768535111) / Math.Sqrt((10717353703.065855 / (299)) + (126674913.30326843 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.13404713635586862 = (2588215.6669628797 - 2241272.768535111) / 2588215.6669628797 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6704581911111112 < 1.8224142990069825.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 105.57413063440515 (T) = (0 -1677942.3956070647) / Math.Sqrt((1190113294.6637723 / (299)) + (19110647.444792278 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14947917120126897 = (1972841.0390336677 - 1677942.3956070647) / 1972841.0390336677 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.593176929653679 < 3.878383713700565.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/16/2022 11:42:33 AM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 24.087944476549158 (T) = (0 -3634049.1551969172) / Math.Sqrt((12390175667.770475 / (299)) + (1598530744.4391778 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.11187287512169519 = (4091811.9190367833 - 3634049.1551969172) / 4091811.9190367833 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.022473627322404 < 2.1709340793723917.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 80.80730062690587 (T) = (0 -2030466.1001038582) / Math.Sqrt((2197534072.1676707 / (299)) + (37131142.14796258 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.13268128641816843 = (2341084.1577699725 - 2030466.1001038582) / 2341084.1577699725 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.0493470567493106 < 2.177463356517857.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 89.27297076570292 (T) = (0 -2054942.5686543933) / Math.Sqrt((1277895131.142775 / (299)) + (30237042.877510406 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12247491618103967 = (2341747.9529033527 - 2054942.5686543933) / 2341747.9529033527 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7570491647887325 < 1.902309743068021.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 35.24617272719694 (T) = (0 -1766392.087586722) / Math.Sqrt((12042300204.528463 / (299)) + (222621576.15485576 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1552253187640928 = (2090962.3912999933 - 1766392.087586722) / 2090962.3912999933 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.072170345238096 < 10.259038429284303.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 15.090217000835988 (T) = (0 -9979707.42626984) / Math.Sqrt((58496868701.587494 / (299)) + (8596455631.800425 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.06206237695058592 = (10640054.49938548 - 9979707.42626984) / 10640054.49938548 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6743674429530204 < 1.8235833439302882.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 53.39728118958582 (T) = (0 -1678228.2508706227) / Math.Sqrt((10591861544.951033 / (299)) + (6690162.254164925 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.16171841932580958 = (2001986.3129056261 - 1678228.2508706227) / 2001986.3129056261 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.244129964242424 < 2.40435333354085.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 38.15880493166032 (T) = (0 -2242239.4680540217) / Math.Sqrt((20036664776.336475 / (299)) + (92272168.97136146 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.13594212343779297 = (2595010.7381407497 - 2242239.4680540217) / 2595010.7381407497 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 20, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToStream - Duration of single invocation 293.16 μs 260.01 μs 0.89 0.02 False
SerializeToUtf8Bytes - Duration of single invocation 242.54 μs 218.89 μs 0.90 0.03 False
SerializeObjectProperty - Duration of single invocation 311.53 μs 288.95 μs 0.93 0.03 False
SerializeToString - Duration of single invocation 310.15 μs 276.83 μs 0.89 0.03 False
SerializeToWriter - Duration of single invocation 287.69 μs 257.54 μs 0.90 0.01 False
SerializeToWriter - Duration of single invocation 238.78 μs 211.16 μs 0.88 0.06 False
SerializeToUtf8Bytes - Duration of single invocation 284.71 μs 254.65 μs 0.89 0.05 False
SerializeToString - Duration of single invocation 255.35 μs 230.15 μs 0.90 0.02 False
SerializeObjectProperty - Duration of single invocation 267.09 μs 237.83 μs 0.89 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToStream(Mode: Reflection)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 260.0062955154477 < 278.921336589072.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 29.721691180772613 (T) = (0 -260377.55211526266) / Math.Sqrt((29069365.645766463 / (112)) + (6236518.647961594 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.12290060883237322 = (296862.08283492085 - 260377.55211526266) / 296862.08283492085 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;.SerializeToUtf8Bytes(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 218.88607002969462 < 227.6102419635876.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 27.295457212312847 (T) = (0 -217292.8544622973) / Math.Sqrt((29755090.907727618 / (112)) + (4576194.646680304 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.12011059135405747 = (246954.73354621715 - 217292.8544622973) / 246954.73354621715 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;.SerializeObjectProperty(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 288.94788625000007 < 296.96069163742237.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 24.045078342812957 (T) = (0 -286326.7455839726) / Math.Sqrt((39150042.98338368 / (112)) + (6992620.723107053 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.09993519728306018 = (318117.92297584057 - 286326.7455839726) / 318117.92297584057 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;.SerializeToString(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 276.8347123276942 < 289.1136215463108.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 22.719471738567435 (T) = (0 -277669.49545962404) / Math.Sqrt((35410056.25621297 / (112)) + (8007711.987773614 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.10177601644471843 = (309131.68713283946 - 277669.49545962404) / 309131.68713283946 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;.SerializeToWriter(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 257.5405705503513 < 273.1489456753246.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 18.684366906627545 (T) = (0 -258022.02762071378) / Math.Sqrt((35843643.8342184 / (112)) + (12334886.417528141 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.10785190156958777 = (289214.3446527108 - 258022.02762071378) / 289214.3446527108 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;.SerializeToWriter(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 211.16426846868887 < 226.80421288431188.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 31.298264214434806 (T) = (0 -214129.4929288263) / Math.Sqrt((26387920.979699865 / (112)) + (3674624.242411121 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.12587058953490685 = (244963.1488945048 - 214129.4929288263) / 244963.1488945048 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;.SerializeToUtf8Bytes(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 254.65298202284947 < 270.70108458578534.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 17.853338170229833 (T) = (0 -256312.52464791498) / Math.Sqrt((48989918.72031543 / (112)) + (14974049.826907095 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.11429477231633758 = (289388.06798988354 - 256312.52464791498) / 289388.06798988354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;.SerializeToString(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 230.15055455179902 < 245.7851704969262.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 21.921408714517852 (T) = (0 -231709.36997261495) / Math.Sqrt((27593738.068569977 / (112)) + (9617418.411118442 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.12231482957682942 = (264000.5525681808 - 231709.36997261495) / 264000.5525681808 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;.SerializeObjectProperty(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 237.83163683712124 < 256.2037500163517.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 22.58268297316476 (T) = (0 -241160.2201715502) / Math.Sqrt((24788248.874538235 / (112)) + (9631379.116754245 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.12066977716561941 = (274254.44265319203 - 241160.2201715502) / 274254.44265319203 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualityOperatorBenchmark - Duration of single invocation 26.66 ns 24.45 ns 0.92 0.18 False
EqualsAllBenchmark - Duration of single invocation 29.50 ns 26.98 ns 0.91 0.24 False
EqualsAnyBenchmark - Duration of single invocation 203.72 ns 184.40 ns 0.91 0.06 False
InequalityOperatorBenchmark - Duration of single invocation 26.95 ns 24.56 ns 0.91 0.29 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.EqualityOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.454414471081396 < 25.330860191894907.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 28.540461249673847 (T) = (0 -24.505793610584156) / Math.Sqrt((1.232675592020165 / (166)) + (0.02173325993169166 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.1121878816675027 = (27.602454510996452 - 24.505793610584156) / 27.602454510996452 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.978735887484017 < 28.047991319558506.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 17.83340637092814 (T) = (0 -27.233640264316914) / Math.Sqrt((2.7621241182593628 / (165)) + (0.05158901843161455 / (5))) is greater than 1.9741851911431834 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (165) + (5) - 2, .975) and 0.09724068947650431 = (30.16711093073586 - 27.233640264316914) / 30.16711093073586 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 184.39901148901325 < 193.74170767200232.
IsChangePoint: Marked as a change because one of 12/2/2022 2:38:36 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 26.380643532582166 (T) = (0 -183.5565351535413) / Math.Sqrt((38.78944928145992 / (165)) + (2.5286961008531144 / (5))) is greater than 1.9741851911431834 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (165) + (5) - 2, .975) and 0.11008370398892335 = (206.2626968135176 - 183.5565351535413) / 206.2626968135176 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;.InequalityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.55662977118709 < 25.244398268215733.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 20.85806803947474 (T) = (0 -24.49949726284359) / Math.Sqrt((2.8997482974998348 / (166)) + (0.03781516833412782 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.11870782903117816 = (27.799517651349166 - 24.49949726284359) / 27.799517651349166 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Strings

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteStringsUtf16 - Duration of single invocation 166.67 ms 142.16 ms 0.85 0.04 False
WriteStringsUtf16 - Duration of single invocation 165.23 ms 141.74 ms 0.86 0.06 False
WriteStringsUtf16 - Duration of single invocation 203.61 ms 186.55 ms 0.92 0.02 False
WriteStringsUtf16 - Duration of single invocation 174.65 ms 148.15 ms 0.85 0.03 False
WriteStringsUtf16 - Duration of single invocation 210.94 ms 193.03 ms 0.92 0.05 False
WriteStringsUtf16 - Duration of single invocation 169.83 ms 148.34 ms 0.87 0.05 False
WriteStringsUtf16 - Duration of single invocation 205.18 ms 186.50 ms 0.91 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Strings*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: True, Escaped: NoneEscaped)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 142.16435993333334 < 158.36353627166667.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/8/2022 11:25:18 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 44.91343787269419 (T) = (0 -142459302.63809523) / Math.Sqrt((70874768853326.42 / (299)) + (1259979297510.5383 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.18064558516416812 = (173867742.77238595 - 142459302.63809523) / 173867742.77238595 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: False, Escaped: NoneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 141.73755946666668 < 157.89712992416665.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/8/2022 6:11:08 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 21.16440275951385 (T) = (0 -144186923.3657143) / Math.Sqrt((80537892652653.58 / (299)) + (10107560032788.527 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.18178147612054318 = (176220556.19331896 - 144186923.3657143) / 176220556.19331896 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: False, Escaped: OneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 186.54812835714287 < 193.271462845.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 34.331254792201655 (T) = (0 -186818081.3586081) / Math.Sqrt((121953657665754.88 / (299)) + (1865551238046.5217 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.13971239865716337 = (217157705.24531654 - 186818081.3586081) / 217157705.24531654 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: False, Escaped: NoneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 148.14614246666667 < 165.27945958282967.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/7/2022 8:57:30 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 35.0386076155214 (T) = (0 -149678884.2304762) / Math.Sqrt((62048228485040.36 / (299)) + (2639327191778.813 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.16718333171400096 = (179726090.9036882 - 149678884.2304762) / 179726090.9036882 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: True, Escaped: OneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 193.02966415384614 < 202.64795388559523.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 38.786076692407896 (T) = (0 -192686226.8879121) / Math.Sqrt((34169592023062.504 / (299)) + (1718273729760.9302 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.11988532798787707 = (218933092.48826838 - 192686226.8879121) / 218933092.48826838 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: True, Escaped: NoneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 148.33535567857143 < 163.3459419147619.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/8/2022 11:25:18 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 37.768767529193504 (T) = (0 -148618774.86238095) / Math.Sqrt((45711019364152.29 / (299)) + (1882867556047.9775 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1560578418625976 = (176100664.5175608 - 148618774.86238095) / 176100664.5175608 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: True, Escaped: OneEscaped)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 186.50147933333335 < 192.95828464166664.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 27.575163212055205 (T) = (0 -184176585.42857143) / Math.Sqrt((113943157180315.08 / (299)) + (4185983425622.5986 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14181935276610788 = (214612839.40882808 - 184176585.42857143) / 214612839.40882808 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.IO.Tests.BinaryWriterExtendedTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteAsciiString - Duration of single invocation 15.19 ms 12.78 ms 0.84 0.11 False
WriteAsciiCharArray - Duration of single invocation 9.63 ms 7.85 ms 0.82 0.15 False
WriteAsciiString - Duration of single invocation 39.65 μs 32.17 μs 0.81 0.12 False
WriteAsciiCharArray - Duration of single invocation 38.78 μs 31.83 μs 0.82 0.13 False
WriteAsciiCharArray - Duration of single invocation 641.46 ns 577.35 ns 0.90 0.12 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 2000000)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.780825082589285 < 14.214458421875.
IsChangePoint: Marked as a change because one of 11/16/2022 11:42:33 AM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 19.307612544083543 (T) = (0 -12919190.440652471) / Math.Sqrt((632307797035.3252 / (299)) + (43078676085.21687 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.13405769462636988 = (14919227.713534793 - 12919190.440652471) / 14919227.713534793 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 2000000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.8483966 < 9.146031551628605.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 42.37965036243819 (T) = (0 -7864181.200968405) / Math.Sqrt((619924615191.4202 / (299)) + (188512073.1648726 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.19846115709683607 = (9811353.835934434 - 7864181.200968405) / 9811353.835934434 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 8000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.16811627492877 < 37.44526281630558.
IsChangePoint: Marked as a change because one of 11/16/2022 11:42:33 AM, 11/17/2022 4:58:45 AM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 39.09302059952187 (T) = (0 -32154.986786681333) / Math.Sqrt((11995272.363909807 / (299)) + (586.9114960387341 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.19605616172162804 = (39996.558535159034 - 32154.986786681333) / 39996.558535159034 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 8000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.829787423625262 < 36.86402159451091.
IsChangePoint: Marked as a change because one of 10/14/2022 6:08:00 AM, 10/14/2022 7:48:13 PM, 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 46.36880734962467 (T) = (0 -31829.707416541158) / Math.Sqrt((6476048.514114705 / (299)) + (22253.683712594826 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.19054140859117555 = (39322.218275728046 - 31829.707416541158) / 39322.218275728046 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 32)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 577.3513561748811 < 613.1913026183729.
IsChangePoint: Marked as a change because one of 10/14/2022 1:19:37 PM, 11/15/2022 8:48:03 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 6.96176096814397 (T) = (0 -579.3695993439958) / Math.Sqrt((345.67240341627894 / (299)) + (162.78924482339934 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.06521955110232903 = (619.7921662003208 - 579.3695993439958) / 619.7921662003208 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 20, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Stack - Duration of single invocation 2.34 ms 736.30 μs 0.32 0.08 False
Span - Duration of single invocation 1.47 ms 697.05 μs 0.47 0.08 False
Array - Duration of single invocation 2.08 ms 735.30 μs 0.35 0.04 False
ICollection - Duration of single invocation 2.07 ms 724.12 μs 0.35 0.02 False
Queue - Duration of single invocation 2.10 ms 746.26 μs 0.36 0.06 False
ImmutableArray - Duration of single invocation 2.10 ms 754.27 μs 0.36 0.05 False
List - Duration of single invocation 2.07 ms 730.76 μs 0.35 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.Stack(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 736.2982642431973 < 2.2415369468418853.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 16.563572088749172 (T) = (0 -732183.580600471) / Math.Sqrt((5577882885722.341 / (299)) + (5660942.51481377 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.7554960407459634 = (2994567.3797443165 - 732183.580600471) / 2994567.3797443165 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Span(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 697.0518518939394 < 1.3982238942541834.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 131.1016925721326 (T) = (0 -697644.5742695982) / Math.Sqrt((3708642779.746599 / (299)) + (117745801.28351897 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.5298058442661941 = (1483737.230167872 - 697644.5742695982) / 1483737.230167872 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 735.3042514880952 < 1.9778439334598215.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 435.56504721376706 (T) = (0 -734342.9695872581) / Math.Sqrt((2819714428.12922 / (299)) + (4368699.759908674 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.6556446590645928 = (2132515.115323863 - 734342.9695872581) / 2132515.115323863 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 724.1228246753246 < 1.9708773852604164.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 464.24257020490023 (T) = (0 -727054.2502846259) / Math.Sqrt((2212385301.522625 / (299)) + (8170287.2434808 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.6574310982159792 = (2122359.1706611225 - 727054.2502846259) / 2122359.1706611225 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 746.2588749999999 < 1.9877250183258928.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 445.5202350826132 (T) = (0 -746271.5391450811) / Math.Sqrt((2852594855.4130898 / (299)) + (1556673.854754446 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.6520311807714412 = (2144650.6063375245 - 746271.5391450811) / 2144650.6063375245 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 754.2667216269841 < 1.9970025881249995.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 463.5992185671161 (T) = (0 -755887.6785214983) / Math.Sqrt((2422167733.153595 / (299)) + (5038279.34897357 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.6492478002404134 = (2155047.5778615233 - 755887.6785214983) / 2155047.5778615233 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 730.76228125 < 1.9615358560221352.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 279.70026662142976 (T) = (0 -732280.683725233) / Math.Sqrt((3689195880.0178924 / (299)) + (62585593.03295644 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.6556786726879668 = (2126736.35247003 - 732280.683725233) / 2126736.35247003 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetFullPathForReallyLongPath - Duration of single invocation 28.44 μs 25.35 μs 0.89 0.05 False
GetFullPathForLegacyLength - Duration of single invocation 5.90 μs 5.12 μs 0.87 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.GetFullPathForReallyLongPath

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.352107843562184 < 27.016248840817294.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 51.478768303753036 (T) = (0 -25341.594233758457) / Math.Sqrt((1158087.6020586723 / (299)) + (29536.52712568876 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.16728959013485264 = (30432.661743549455 - 25341.594233758457) / 30432.661743549455 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_Path.GetFullPathForLegacyLength

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.121171008829657 < 5.620514346621362.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/23/2022 8:43:59 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 61.01328606552141 (T) = (0 -5161.604558684108) / Math.Sqrt((38876.26533265101 / (299)) + (830.6248363600109 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.16903456614135332 = (6211.57553415409 - 5161.604558684108) / 6211.57553415409 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfString - Duration of single invocation 1.58 μs 1.37 μs 0.87 0.05 False
IndexOfString - Duration of single invocation 1.57 μs 1.37 μs 0.87 0.03 False
IndexOfString - Duration of single invocation 629.56 ns 542.09 ns 0.86 0.05 False
IndexOfString - Duration of single invocation 226.52 ns 202.50 ns 0.89 0.13 False
IndexOfString - Duration of single invocation 222.93 ns 191.39 ns 0.86 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "X", comparisonType: Ordinal)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3670244941415381 < 1.4923736923217528.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 73.96014497836661 (T) = (0 -1369.1402846550664) / Math.Sqrt((2958.2620523273436 / (299)) + (6.056930814359306 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.15255465424727724 = (1615.6089493169152 - 1369.1402846550664) / 1615.6089493169152 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだxだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "x", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.365139610649767 < 1.4955482540920795.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 105.54304044147165 (T) = (0 -1368.8484075660713) / Math.Sqrt((1173.966687884611 / (299)) + (7.331196382588326 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.15185865273884186 = (1613.9390114470837 - 1368.8484075660713) / 1613.9390114470837 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 542.0922904486822 < 598.4752379893694.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 61.51810596108842 (T) = (0 -542.3160467103274) / Math.Sqrt((952.0023601794867 / (299)) + (0.1825085679817693 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.16913699622380446 = (652.7141589474452 - 542.3160467103274) / 652.7141589474452 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 202.49731114325206 < 214.43819857932996.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 11.546990525774765 (T) = (0 -193.00364005227433) / Math.Sqrt((138.32068419853272 / (299)) + (37.78232946557871 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14487547140236678 = (225.70237853987402 - 193.00364005227433) / 225.70237853987402 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 191.38793050080858 < 212.54582430677792.
IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/11/2022 10:51:47 AM, 11/15/2022 5:29:43 PM, 12/2/2022 10:04:21 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 37.18238492576849 (T) = (0 -191.1589705991502) / Math.Sqrt((130.18854524649595 / (299)) + (1.4246190006375998 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14169431005258415 = (222.71665309694214 - 191.1589705991502) / 222.71665309694214 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Hashtable>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToStream - Duration of single invocation 289.72 μs 260.93 μs 0.90 0.03 False
SerializeObjectProperty - Duration of single invocation 309.20 μs 289.39 μs 0.94 0.03 False
SerializeToUtf8Bytes - Duration of single invocation 279.37 μs 259.52 μs 0.93 0.04 False
SerializeToStream - Duration of single invocation 288.89 μs 265.72 μs 0.92 0.03 False
SerializeToString - Duration of single invocation 302.94 μs 278.76 μs 0.92 0.00 False
SerializeToWriter - Duration of single invocation 281.17 μs 259.09 μs 0.92 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Hashtable&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<Hashtable>.SerializeToStream(Mode: Reflection)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 260.92856496527776 < 272.859310263934.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 23.538210233124033 (T) = (0 -264557.1786296576) / Math.Sqrt((36479149.38319566 / (112)) + (6505593.435821002 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.10191577775327666 = (294579.47492699407 - 264557.1786296576) / 294579.47492699407 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Hashtable&gt;.SerializeObjectProperty(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 289.3869336038961 < 294.7239853482886.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 8.015872919875514 (T) = (0 -292345.2581200704) / Math.Sqrt((44396269.00214754 / (112)) + (47409342.05912111 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.07934036249435435 = (317538.91037530976 - 292345.2581200704) / 317538.91037530976 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Hashtable&gt;.SerializeToUtf8Bytes(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 259.5222176912568 < 266.8630884123884.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 25.028490783881676 (T) = (0 -259653.3366252677) / Math.Sqrt((41042236.19548574 / (112)) + (5015163.90758387 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.10136796069765942 = (288942.8879331427 - 259653.3366252677) / 288942.8879331427 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Hashtable&gt;.SerializeToStream(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 265.71830652777777 < 272.34491627407675.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 26.56484854681995 (T) = (0 -265606.4705063559) / Math.Sqrt((42504445.716943756 / (112)) + (3853686.909257881 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.09687486553433716 = (294097.08618452185 - 265606.4705063559) / 294097.08618452185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Hashtable&gt;.SerializeToString(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 278.7555967793367 < 287.91143631274036.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 15.39221585474959 (T) = (0 -278747.55643252505) / Math.Sqrt((33891355.672897 / (112)) + (18799109.032819916 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.10015037875086769 = (309771.2660539655 - 278747.55643252505) / 309771.2660539655 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Hashtable&gt;.SerializeToWriter(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 259.090855942623 < 265.5475772743056.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 15.042541959955447 (T) = (0 -258007.21963498686) / Math.Sqrt((35699005.53315568 / (112)) + (18802584.329171665 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.10534969301831489 = (288388.9019224007 - 258007.21963498686) / 288388.9019224007 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 20, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MatchesWord - Duration of single invocation 39.63 μs 33.69 μs 0.85 0.03 False
MatchesWord - Duration of single invocation 39.90 μs 33.51 μs 0.84 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: None)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.694687879609546 < 37.70863800147742.
IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 11/15/2022 8:48:03 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 24.069501312252648 (T) = (0 -34058.23328917286) / Math.Sqrt((1758876.6118471588 / (299)) + (104572.56120623977 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.10369209250103431 = (37998.36306722772 - 34058.23328917286) / 37998.36306722772 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.51033016577061 < 37.98502890410125.
IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 11/16/2022 11:42:33 AM, 12/2/2022 5:21:30 AM, 12/7/2022 5:17:51 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 20.62909827084111 (T) = (0 -33721.70454213086) / Math.Sqrt((1768308.3662038066 / (299)) + (187730.83734683192 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.11310666298188192 = (38022.27746518967 - 33721.70454213086) / 38022.27746518967 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Tests.Perf_Environment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetEnvironmentVariable - Duration of single invocation 636.00 ns 561.01 ns 0.88 0.06 False
SetEnvironmentVariable - Duration of single invocation 1.03 μs 828.22 ns 0.81 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Environment*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Environment.GetEnvironmentVariable

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 561.0053371102626 < 604.5266276950359.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 14.457798132893062 (T) = (0 -561.8776771707647) / Math.Sqrt((484.66470882872846 / (299)) + (60.47321310044732 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.0870035165584907 = (615.4215129644157 - 561.8776771707647) / 615.4215129644157 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Environment.SetEnvironmentVariable

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 828.2209565789474 < 0.977082209064075.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 48.76812326790563 (T) = (0 -820.8876600365095) / Math.Sqrt((1267.4442371481573 / (299)) + (56.74486446500583 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.18999212385759295 = (1013.4317013630985 - 820.8876600365095) / 1013.4317013630985 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SystemTextJson_SourceGen_ - Duration of single invocation 576.68 μs 532.24 μs 0.92 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.SystemTextJson_SourceGen_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 532.24042700491 < 545.7612046899072.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 34.6369118238132 (T) = (0 -534547.9339444734) / Math.Sqrt((120732007.36591357 / (112)) + (5879285.7701296285 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.0886534836208142 = (586547.4046779183 - 534547.9339444734) / 586547.4046779183 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 19.47 ms 17.14 ms 0.88 0.11 False
Count - Duration of single invocation 19.15 ms 16.77 ms 0.88 0.09 False
Count - Duration of single invocation 19.50 ms 17.12 ms 0.88 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w\.+-]+@[\w\.-]+\.[\w\.-]+", Options: None)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.136989005494506 < 18.52031182409746.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 92.720358919832 (T) = (0 -17149149.36709053) / Math.Sqrt((281861780567.70856 / (291)) + (411887019.55001104 / (5))) is greater than 1.9680656892939519 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (291) + (5) - 2, .975) and 0.14913783700973696 = (20155026.410883874 - 17149149.36709053) / 20155026.410883874 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\\w\\.+-]+@[\\w\\.-]+\\.[\\w\\.-]+", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.768266959183673 < 18.21190965313645.
IsChangePoint: Marked as a change because one of 11/18/2022 3:21:53 PM, 11/19/2022 1:29:52 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 55.25183366923394 (T) = (0 -16778387.63170975) / Math.Sqrt((873180706547.0952 / (292)) + (558404475.4730942 / (5))) is greater than 1.9680381148904758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (292) + (5) - 2, .975) and 0.15498316777104026 = (19855684.516309846 - 16778387.63170975) / 19855684.516309846 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\\w\\.+-]+@[\\w\\.-]+\\.[\\w\\.-]+", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.12267338095238 < 18.522512875032053.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 109.84631986924009 (T) = (0 -17151519.133809526) / Math.Sqrt((164332379409.9132 / (289)) + (750747385.8552369 / (5))) is greater than 1.9681214070095314 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (289) + (5) - 2, .975) and 0.1465417193823608 = (20096493.904068917 - 17151519.133809526) / 20096493.904068917 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Replace_Char - Duration of single invocation 61.77 ns 51.89 ns 0.84 0.29 False
PadLeft - Duration of single invocation 7.19 μs 5.70 μs 0.79 0.08 False
Replace_Char - Duration of single invocation 146.43 ns 106.32 ns 0.73 0.12 False
Replace_String - Duration of single invocation 204.44 ns 175.45 ns 0.86 0.22 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Replace_Char(text: "Hello", oldChar: 'a', newChar: 'b')

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.88716086842773 < 58.10505244473926.
IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 12.89844003144573 (T) = (0 -47.764930630981624) / Math.Sqrt((12.132635439081168 / (299)) + (7.219890939424654 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.24756757754746428 = (63.480691694933824 - 47.764930630981624) / 63.480691694933824 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.PadLeft(n: 2142)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.704061058424741 < 6.827698501911253.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 127.84157653816759 (T) = (0 -5711.483206210671) / Math.Sqrt((42168.93027854554 / (299)) + (185.56905790764822 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.230030743593185 = (7417.8068262935885 - 5711.483206210671) / 7417.8068262935885 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'z', newChar: 'y')

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 106.32160745021143 < 132.87489649086515.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 29.098369923362704 (T) = (0 -103.83013783411798) / Math.Sqrt((21.573737833113118 / (299)) + (6.707810212603009 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.24993413158446345 = (138.42802639913762 - 103.83013783411798) / 138.42802639913762 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "bad", newValue: "nice")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 175.4455740763585 < 200.76558482243814.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 18.824027351752836 (T) = (0 -177.3540962511889) / Math.Sqrt((101.13357803742556 / (299)) + (6.974215084255535 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12259704969992638 = (202.13528594875757 - 177.3540962511889) / 202.13528594875757 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 28.48 ns 26.73 ns 0.94 0.24 False
LessThanAllBenchmark - Duration of single invocation 27.79 ns 25.89 ns 0.93 0.27 False
EqualityOperatorBenchmark - Duration of single invocation 29.21 ns 25.20 ns 0.86 0.24 False
EqualsAllBenchmark - Duration of single invocation 29.87 ns 27.98 ns 0.94 0.24 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GreaterThanOrEqualAnyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.734801703634933 < 27.675396890607225.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 21.65202197805247 (T) = (0 -26.600156317984226) / Math.Sqrt((2.8349912071508165 / (166)) + (0.06143729382680841 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.12241231481427016 = (30.31053963838913 - 26.600156317984226) / 30.31053963838913 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.LessThanAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.885684537129123 < 26.377531984807117.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 19.421800598779512 (T) = (0 -25.750026865774736) / Math.Sqrt((3.374106835032144 / (166)) + (0.09156390276974863 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.12911689286539815 = (29.56771885322018 - 25.750026865774736) / 29.56771885322018 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.198396489710053 < 26.913209148927397.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 6.118580719916153 (T) = (0 -25.69817627178096) / Math.Sqrt((2.8074824715846165 / (166)) + (1.7693116838403244 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.1266211397707349 = (29.42385881086599 - 25.69817627178096) / 29.42385881086599 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.984217376097547 < 28.38065529567089.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 18.652631414442745 (T) = (0 -27.70248663615314) / Math.Sqrt((2.9042901701929873 / (166)) + (0.14751514748315553 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.12737704181726703 = (31.746227137828818 - 27.70248663615314) / 31.746227137828818 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SystemTextJson_SourceGen_ - Duration of single invocation 437.34 μs 390.28 μs 0.89 0.01 False
SystemTextJson_Reflection_ - Duration of single invocation 583.43 μs 531.34 μs 0.91 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>.SystemTextJson_SourceGen_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 390.28276918859643 < 415.08275786158646.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 16.475401340061957 (T) = (0 -394258.67817419535) / Math.Sqrt((74951515.64513883 / (112)) + (50023970.90888067 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.1201264608015053 = (448085.6175460605 - 394258.67817419535) / 448085.6175460605 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### MicroBenchmarks.Serializers.Json_ToString&lt;IndexViewModel&gt;.SystemTextJson_Reflection_

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 531.3444589578871 < 553.6947529825497.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 14.96259633875018 (T) = (0 -540167.5429827275) / Math.Sqrt((75660446.2493648 / (112)) + (61824446.68480829 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.09093276930781036 = (594199.7739500835 - 540167.5429827275) / 594199.7739500835 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualityOperatorBenchmark - Duration of single invocation 21.16 ns 18.41 ns 0.87 0.19 False
EqualsAllBenchmark - Duration of single invocation 21.27 ns 19.79 ns 0.93 0.25 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int16>.EqualityOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.41023640976456 < 20.011128254225234.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/14/2022 1:19:37 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 29.889708926014013 (T) = (0 -18.63465839836133) / Math.Sqrt((1.6807942851334623 / (299)) + (0.03915899738986005 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.15686008850799443 = (22.101501950471977 - 18.63465839836133) / 22.101501950471977 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.79079777701875 < 20.075593148692793.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/14/2022 7:48:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 4.341007312626844 (T) = (0 -19.566995046447758) / Math.Sqrt((1.5235897429482053 / (299)) + (1.811876235463876 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1185437351131239 = (22.19848655674248 - 19.566995046447758) / 22.19848655674248 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 20, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 26.80 secs 23.11 secs 0.86 0.09 False
Count - Duration of single invocation 26.78 secs 22.71 secs 0.85 0.06 False
Count - Duration of single invocation 26.51 secs 22.79 secs 0.86 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: NonBacktracking)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.105596826333333 < 25.495111391208333.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/16/2022 11:42:33 AM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 25.385484599571075 (T) = (0 -23345781597.466667) / Math.Sqrt((8.29983237589385E+17 / (299)) + (95245594863821840 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1384064783101048 = (27096050527.024834 - 23345781597.466667) / 27096050527.024834 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.714905501666667 < 25.27286940929167.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 66.10789560675495 (T) = (0 -22827168479.06667) / Math.Sqrt((4.072703484156316E+17 / (299)) + (10107124154400748 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14416974587760428 = (26672542094.78094 - 22827168479.06667) / 26672542094.78094 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.793677556333332 < 25.240131903574998.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 40.59351145631759 (T) = (0 -22880815787.8) / Math.Sqrt((5.179886402794731E+17 / (299)) + (36155862906997216 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14381193780081938 = (26724053742.38573 - 22880815787.8) / 26724053742.38573 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualityOperatorBenchmark - Duration of single invocation 19.97 ns 17.51 ns 0.88 0.31 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<SByte>.EqualityOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.505288129543597 < 19.566352890320395.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/22/2022 2:20:42 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 10.320915956784622 (T) = (0 -18.22484525485214) / Math.Sqrt((1.616744480322363 / (299)) + (0.24923353529142964 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.11747914002573367 = (20.650894592319965 - 18.22484525485214) / 20.650894592319965 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsAnyBenchmark - Duration of single invocation 207.59 ns 184.65 ns 0.89 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.EqualsAnyBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 184.65492443417915 < 196.03884458873992.
IsChangePoint: Marked as a change because one of 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.678556994565927 (T) = (0 -188.41486352357975) / Math.Sqrt((42.18642534960816 / (165)) + (27.613105199542993 / (5))) is greater than 1.9741851911431834 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (165) + (5) - 2, .975) and 0.08922292343407842 = (206.87264575651886 - 188.41486352357975) / 206.87264575651886 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SystemTextJson_SourceGen_ - Duration of single invocation 634.29 μs 596.96 μs 0.94 0.01 False
SystemTextJson_Reflection_ - Duration of single invocation 618.06 μs 573.48 μs 0.93 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>.SystemTextJson_SourceGen_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 596.9621181089744 < 603.1404413474361.
IsChangePoint: Marked as a change because one of 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 19.72229367819841 (T) = (0 -597944.66385561) / Math.Sqrt((110309145.54238461 / (112)) + (21470818.386562087 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.07044498822437693 = (643259.03930465 - 597944.66385561) / 643259.03930465 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### MicroBenchmarks.Serializers.Json_FromString&lt;IndexViewModel&gt;.SystemTextJson_Reflection_

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 573.4782686075499 < 587.2920934254807.
IsChangePoint: Marked as a change because one of 12/8/2022 9:11:55 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 27.153292838846593 (T) = (0 -577502.6279477004) / Math.Sqrt((111402109.6279173 / (112)) + (10419981.309260856 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.0762117108747703 = (625146.0802718766 - 577502.6279477004) / 625146.0802718766 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOf - Duration of single invocation 77.31 ns 68.53 ns 0.89 0.17 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.StringSegmentBenchmark.LastIndexOf

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 68.53219739960842 < 73.7684490517314.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 22.104746171009378 (T) = (0 -69.11025283166808) / Math.Sqrt((11.003105659956585 / (299)) + (1.0853582321707724 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.13879021154528637 = (80.24787195658101 - 69.11025283166808) / 80.24787195658101 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableHashSet - Duration of single invocation 187.00 μs 171.75 μs 0.92 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<String>.ImmutableHashSet(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 171.75372414638932 < 177.77821841942347.
IsChangePoint: Marked as a change because one of 10/5/2022 3:28:28 PM, 11/15/2022 9:22:46 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 19.099812949625807 (T) = (0 -171958.65560311815) / Math.Sqrt((15201324.85696605 / (299)) + (2756099.4297272777 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.07934541537756486 = (186778.68820219825 - 171958.65560311815) / 186778.68820219825 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
InequalityOperatorBenchmark - Duration of single invocation 27.90 ns 26.08 ns 0.93 0.25 False
EqualsAnyBenchmark - Duration of single invocation 88.30 ns 82.57 ns 0.94 0.18 False
LessThanOrEqualAllBenchmark - Duration of single invocation 28.69 ns 26.92 ns 0.94 0.20 False
EqualityOperatorBenchmark - Duration of single invocation 27.86 ns 25.92 ns 0.93 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.InequalityOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.076680773546055 < 26.5164609980363.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 11.30617144473641 (T) = (0 -25.512793214023382) / Math.Sqrt((4.991729325272642 / (165)) + (0.5151954458797716 / (5))) is greater than 1.9741851911431834 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (165) + (5) - 2, .975) and 0.13926143119978038 = (29.64058326047312 - 25.512793214023382) / 29.64058326047312 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 82.57348481050187 < 83.81176099547484.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 20.532120212424125 (T) = (0 -82.70995327474404) / Math.Sqrt((16.318025990493155 / (166)) + (0.3532576863965573 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.09258967565155922 = (91.14945141728832 - 82.70995327474404) / 91.14945141728832 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.LessThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.92269051319944 < 27.057192337157506.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 22.829971732108838 (T) = (0 -26.674970399189704) / Math.Sqrt((1.563555555714125 / (166)) + (0.0607373428928807 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.11165374999106695 = (30.027672654577497 - 26.674970399189704) / 30.027672654577497 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.919114200302282 < 26.463205944350484.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 22.00150877122351 (T) = (0 -25.66854204045045) / Math.Sqrt((1.898980995464527 / (166)) + (0.07512052130981153 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.1223733249082792 = (29.247677593400212 - 25.66854204045045) / 29.247677593400212 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOf_Word_NotFound - Duration of single invocation 935.79 ns 797.42 ns 0.85 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, Ordinal, False))

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 797.4195161662036 < 892.4138671212169.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 45.54528550753214 (T) = (0 -803.2367466861417) / Math.Sqrt((1132.4311186853167 / (299)) + (31.824204620293955 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1530216920447408 = (948.3557478883768 - 803.2367466861417) / 948.3557478883768 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableList - Duration of single invocation 8.49 ms 7.65 ms 0.90 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<String>.ImmutableList(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.649560363095238 < 8.01103801455729.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 27.2326038683267 (T) = (0 -7539016.992519842) / Math.Sqrt((62509511301.23496 / (299)) + (4482164343.534992 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1072247266912718 = (8444473.338267284 - 7539016.992519842) / 8444473.338267284 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Collections.ContainsKeyFalse<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableDictionary - Duration of single invocation 185.98 μs 168.51 μs 0.91 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<String, String>.ImmutableDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 168.50654643173758 < 174.9074152174224.
IsChangePoint: Marked as a change because one of 11/30/2022 11:21:33 PM, 12/2/2022 12:08:23 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 18.6449144833316 (T) = (0 -168572.70013260475) / Math.Sqrt((18239167.9480073 / (299)) + (2668901.390208378 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.07859621455785759 = (182952.0377450087 - 168572.70013260475) / 182952.0377450087 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsAllBenchmark - Duration of single invocation 21.02 ns 18.55 ns 0.88 0.27 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt64>.EqualsAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.552650868425122 < 20.025743587714633.
IsChangePoint: Marked as a change because one of 10/15/2022 9:29:34 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 19.780499921103996 (T) = (0 -18.900894632360696) / Math.Sqrt((2.61414937920987 / (299)) + (0.11028806844396465 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1551687849977579 = (22.372391427689536 - 18.900894632360696) / 22.372391427689536 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Convert_uint_float - Duration of single invocation 130.23 μs 105.28 μs 0.81 0.21 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorConvert.Convert_uint_float

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 105.27623969122807 < 124.64437819122392.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/17/2022 12:47:34 PM, 12/3/2022 2:55:34 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 21.050447137234322 (T) = (0 -105923.30176275884) / Math.Sqrt((123891933.71582244 / (299)) + (8058355.060884201 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.2204999105065403 = (135886.19576886858 - 105923.30176275884) / 135886.19576886858 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 20, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyFiveValues - Duration of single invocation 96.37 ns 73.89 ns 0.77 0.38 False
IndexOfValue - Duration of single invocation 794.51 ns 685.80 ns 0.86 0.06 False
IndexOfValue - Duration of single invocation 41.85 ns 38.19 ns 0.91 0.29 False
LastIndexOfValue - Duration of single invocation 84.56 ns 76.28 ns 0.90 0.08 False
LastIndexOfAnyValues - Duration of single invocation 88.81 ns 39.06 ns 0.44 0.14 False
LastIndexOfValue - Duration of single invocation 46.68 ns 34.08 ns 0.73 0.17 False
IndexOfValue - Duration of single invocation 87.38 ns 73.49 ns 0.84 0.19 False
LastIndexOfAnyValues - Duration of single invocation 3.79 μs 1.00 μs 0.26 0.10 False
IndexOfAnyFiveValues - Duration of single invocation 301.51 ns 231.76 ns 0.77 0.14 False
LastIndexOfAnyValues - Duration of single invocation 329.75 ns 96.23 ns 0.29 0.09 False
Fill - Duration of single invocation 134.65 ns 117.79 ns 0.87 0.16 False
Fill - Duration of single invocation 1.68 μs 1.32 μs 0.79 0.16 False
SequenceCompareTo - Duration of single invocation 1.15 μs 938.09 ns 0.81 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.IndexOfAnyFiveValues(Size: 4)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 73.89304332200143 < 86.43150146879205.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 14.737321050090044 (T) = (0 -75.9318185684007) / Math.Sqrt((19.985404325031638 / (70)) + (4.483789070134597 / (5))) is greater than 1.9929971258892505 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (5) - 2, .975) and 0.17425917632075813 = (91.9559944124748 - 75.9318185684007) / 91.9559944124748 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 685.7961618613828 < 757.0238811179598.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 35.46880274521712 (T) = (0 -690.5658348920489) / Math.Sqrt((939.1530082793757 / (299)) + (48.81745624760944 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.15576652032053018 = (817.9796839545336 - 690.5658348920489) / 817.9796839545336 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfValue(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.19456111328507 < 39.68137252837474.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 9.77850932548138 (T) = (0 -36.67257712398754) / Math.Sqrt((3.0845822037683637 / (70)) + (1.2149126229074279 / (5))) is greater than 1.9929971258892505 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (5) - 2, .975) and 0.1250016407085654 = (41.9115953013725 - 36.67257712398754) / 41.9115953013725 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.LastIndexOfValue(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 76.28372610325683 < 81.20760511045421.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 13.2568125323891 (T) = (0 -73.84171817305136) / Math.Sqrt((5.443063106093785 / (70)) + (3.8163655112815493 / (5))) is greater than 1.9929971258892505 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (5) - 2, .975) and 0.14136780921396877 = (85.99924270886382 - 73.84171817305136) / 85.99924270886382 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.LastIndexOfAnyValues(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 39.0561697716315 < 85.59736583290248.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 125.55277100859729 (T) = (0 -39.13529096319055) / Math.Sqrt((10.164990768391144 / (70)) + (0.1068811961942263 / (5))) is greater than 1.9929971258892505 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (5) - 2, .975) and 0.5669929613662305 = (90.38026514920132 - 39.13529096319055) / 90.38026514920132 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.LastIndexOfValue(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.07918420869989 < 44.37235546688227.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 43.35068070987827 (T) = (0 -34.277033036287534) / Math.Sqrt((6.053923951535513 / (70)) + (0.10743441863962716 / (5))) is greater than 1.9929971258892505 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (5) - 2, .975) and 0.2935722684338954 = (48.52164135784643 - 34.277033036287534) / 48.52164135784643 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfValue(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 73.49188328503267 < 83.33888701333895.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 22.492438378559203 (T) = (0 -72.12065660319567) / Math.Sqrt((9.635282410323951 / (70)) + (1.357387382930984 / (5))) is greater than 1.9929971258892505 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (5) - 2, .975) and 0.1663070532341443 = (86.50745683164679 - 72.12065660319567) / 86.50745683164679 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.LastIndexOfAnyValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0019076606597617 < 3.574757070315623.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 295.91653974418483 (T) = (0 -1001.0374204479391) / Math.Sqrt((28105.41880596666 / (299)) + (0.8788790420263135 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.7415147590165185 = (3872.706297037324 - 1001.0374204479391) / 3872.706297037324 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyFiveValues(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 231.76488309704706 < 281.3531702197276.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 44.866071312612085 (T) = (0 -234.04641482464984) / Math.Sqrt((49.161991684362555 / (70)) + (6.4017134585890405 / (5))) is greater than 1.9929971258892505 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (5) - 2, .975) and 0.2125506573961735 = (297.220915888682 - 234.04641482464984) / 297.220915888682 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.LastIndexOfAnyValues(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 96.23022692937275 < 304.5899414006463.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 81.58061657942476 (T) = (0 -99.61360033560534) / Math.Sqrt((29.117482367964065 / (70)) + (35.2441991093315 / (5))) is greater than 1.9929971258892505 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (5) - 2, .975) and 0.6911268197088233 = (322.5064741512972 - 99.61360033560534) / 322.5064741512972 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.Fill(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 117.78891552459085 < 128.41532893192283.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 35.01939813818087 (T) = (0 -118.27325900164517) / Math.Sqrt((17.029402770113684 / (70)) + (0.4840292018726195 / (5))) is greater than 1.9929971258892505 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (5) - 2, .975) and 0.14724451084449552 = (138.6953945248395 - 118.27325900164517) / 138.6953945248395 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.Fill(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3197489850655537 < 1.5913859127113075.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/6/2022 10:14:50 PM, 10/7/2022 12:26:39 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 45.36384714744791 (T) = (0 -1320.6296934259562) / Math.Sqrt((25607.058308378062 / (299)) + (2.041566668000465 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.24164526850319634 = (1741.440566764002 - 1320.6296934259562) / 1741.440566764002 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.SequenceCompareTo(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 938.0929048988161 < 1.0965995121662329.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 63.72367600431787 (T) = (0 -940.2420494447072) / Math.Sqrt((2980.221460169992 / (299)) + (6.124698494374118 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.18482836832963678 = (1153.428324680611 - 940.2420494447072) / 1153.428324680611 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
InequalityOperatorBenchmark - Duration of single invocation 27.72 ns 25.85 ns 0.93 0.20 False
EqualsAllBenchmark - Duration of single invocation 31.03 ns 28.57 ns 0.92 0.18 False
EqualsAnyBenchmark - Duration of single invocation 88.60 ns 81.54 ns 0.92 0.21 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.InequalityOperatorBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.84785852205074 < 26.69174843748149.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 12.775376951673692 (T) = (0 -25.534620455721587) / Math.Sqrt((1.9317352024858456 / (165)) + (0.3954546714671153 / (5))) is greater than 1.9741851911431834 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (165) + (5) - 2, .975) and 0.13100837345305902 = (29.384196205879395 - 25.534620455721587) / 29.384196205879395 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.571032481035335 < 28.78477817882065.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 8.416736734717697 (T) = (0 -28.68845953964675) / Math.Sqrt((1.2288631241356298 / (166)) + (0.519785974232421 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.0891736759119051 = (31.49717874960321 - 28.68845953964675) / 31.49717874960321 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 81.54269828987117 < 84.47399250124022.
IsChangePoint: Marked as a change because one of 12/18/2022 12:43:31 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 23.31056108333767 (T) = (0 -82.36288832970384) / Math.Sqrt((9.465402164511007 / (167)) + (0.3342298505427384 / (4))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (167) + (4) - 2, .975) and 0.09583006447291101 = (91.09226605913423 - 82.36288832970384) / 91.09226605913423 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DotBenchmark - Duration of single invocation 142.74 ns 131.22 ns 0.92 0.17 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 28.52 ns 26.90 ns 0.94 0.28 False
EqualityOperatorBenchmark - Duration of single invocation 27.83 ns 25.71 ns 0.92 0.18 False
GreaterThanOrEqualAllBenchmark - Duration of single invocation 141.84 ns 130.24 ns 0.92 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.DotBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 131.22236696650464 < 135.85323210443755.
IsChangePoint: Marked as a change because one of 11/22/2022 11:24:47 AM, 12/1/2022 11:48:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 29.15656551680956 (T) = (0 -132.47741723430647) / Math.Sqrt((49.82811441728074 / (166)) + (0.5923029421406583 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.12464977764497337 = (151.34218721952408 - 132.47741723430647) / 151.34218721952408 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.895935856693086 < 27.091736909923956.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 15.39900498324651 (T) = (0 -26.57690665109584) / Math.Sqrt((3.1003052259698913 / (166)) + (0.21295239368962582 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.12542867323431955 = (30.388495297898622 - 26.57690665109584) / 30.388495297898622 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.705952226392046 < 26.416283028089442.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 18.139427992734007 (T) = (0 -25.728379332614388) / Math.Sqrt((1.3571716207182323 / (166)) + (0.1548397060806841 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.12241413533411237 = (29.31722167426846 - 25.728379332614388) / 29.31722167426846 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;.GreaterThanOrEqualAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 130.24304531089373 < 134.66605493405245.
IsChangePoint: Marked as a change because one of 11/22/2022 11:24:47 AM, 12/1/2022 11:48:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 22.814174197489272 (T) = (0 -130.5771941904791) / Math.Sqrt((37.10235558852038 / (166)) + (0.7831981041555456 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.09724829009978028 = (144.64353017388544 - 130.5771941904791) / 144.64353017388544 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsStaticBenchmark - Duration of single invocation 75.95 ns 67.34 ns 0.89 0.14 False
MaxBenchmark - Duration of single invocation 88.77 ns 82.46 ns 0.93 0.19 False
EqualityOperatorBenchmark - Duration of single invocation 21.29 ns 19.03 ns 0.89 0.30 False
EqualsAllBenchmark - Duration of single invocation 22.00 ns 19.31 ns 0.88 0.26 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int32>.EqualsStaticBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 67.33728917073687 < 72.59405311353413.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 38.84754842212528 (T) = (0 -67.46805732137742) / Math.Sqrt((25.65607560017591 / (299)) + (0.10371103496810863 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1582127207606494 = (80.14858264708204 - 67.46805732137742) / 80.14858264708204 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.MaxBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 82.45894868120529 < 84.24895106140615.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 28.90645439530844 (T) = (0 -82.67884472920541) / Math.Sqrt((15.959164899155516 / (299)) + (0.34042384149977134 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.10861330651335158 = (92.75306141917835 - 82.67884472920541) / 92.75306141917835 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.02860597318271 < 19.988194477493707.
IsChangePoint: Marked as a change because one of 10/14/2022 1:19:37 PM, 12/18/2022 12:43:31 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 5.086291167220334 (T) = (0 -18.17910499724921) / Math.Sqrt((2.342547406988018 / (299)) + (2.6614682637304665 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.18670271576921146 = (22.3523493189122 - 18.17910499724921) / 22.3523493189122 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.311214815946506 < 20.00999794146295.
IsChangePoint: Marked as a change because one of 10/14/2022 3:33:41 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 18.141955829267385 (T) = (0 -18.878333908562176) / Math.Sqrt((2.1691195225885678 / (299)) + (0.11009058221254578 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14120255430564627 = (21.98228930839296 - 18.878333908562176) / 21.98228930839296 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 20, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetString - Duration of single invocation 2.30 ms 2.07 ms 0.90 0.03 False
GetString - Duration of single invocation 1.51 ms 1.37 ms 0.91 0.02 False
GetBytes - Duration of single invocation 1.26 ms 1.11 ms 0.89 0.13 False
GetString - Duration of single invocation 1.58 ms 1.42 ms 0.90 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Utf8Encoding.GetString(Input: Greek)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.0692476958333335 < 2.1443009000478312.
IsChangePoint: Marked as a change because one of 10/12/2022 11:17:15 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 18.67563735179074 (T) = (0 -2036092.636873283) / Math.Sqrt((1227664868.3063555 / (299)) + (681973340.0360081 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.09806055018199 = (2257460.4506811607 - 2036092.636873283) / 2257460.4506811607 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetString(Input: EnglishMostlyAscii)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.372429047247024 < 1.4363495297463473.
IsChangePoint: Marked as a change because one of 10/5/2022 12:22:07 PM, 11/16/2022 11:42:33 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 26.28891356806199 (T) = (0 -1372358.8589483434) / Math.Sqrt((666628588.0323737 / (299)) + (105845616.4516735 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.08480366548037269 = (1499523.98975535 - 1372358.8589483434) / 1499523.98975535 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishAllAscii)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1115749361263738 < 1.194683273546906.
IsChangePoint: Marked as a change because one of 11/1/2022 1:15:18 PM, 11/1/2022 10:53:48 PM, 11/16/2022 11:42:33 AM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 35.163556136261974 (T) = (0 -1107103.3439862966) / Math.Sqrt((6122008859.469288 / (299)) + (12967201.582180198 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.13235895340665907 = (1275992.3569005495 - 1107103.3439862966) / 1275992.3569005495 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetString(Input: Cyrillic)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.4174629170454547 < 1.5051139139285714.
IsChangePoint: Marked as a change because one of 10/14/2022 9:06:25 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 24.5735268546121 (T) = (0 -1431380.811105769) / Math.Sqrt((432153172.74796003 / (299)) + (178120736.24236053 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.09463350798171673 = (1580996.009599241 - 1431380.811105769) / 1580996.009599241 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Globalization.Tests.StringEquality

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Compare_Same_Upper - Duration of single invocation 322.24 ns 287.67 ns 0.89 0.07 False
Compare_Same - Duration of single invocation 2.28 μs 1.92 μs 0.84 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringEquality*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, Ordinal))

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 287.66782111019114 < 306.6498104671918.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 11.299103011683862 (T) = (0 -295.40485072007704) / Math.Sqrt((102.45816371643853 / (299)) + (52.80890467548059 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.11214279114943458 = (332.7166212937698 - 295.40485072007704) / 332.7166212937698 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringEquality.Compare_Same(Count: 1024, Options: (en-US, Ordinal))

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.924074923178421 < 2.166722523094706.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 59.06314199454337 (T) = (0 -1928.0591936341893) / Math.Sqrt((12382.777673856921 / (299)) + (13.092200156964623 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.16893428188345477 = (2319.9840296670814 - 1928.0591936341893) / 2319.9840296670814 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Memory.SequenceReader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryReadTo - Duration of single invocation 5.24 μs 2.00 μs 0.38 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.SequenceReader*'

Payloads

Baseline
Compare

Histogram

System.Memory.SequenceReader.TryReadTo

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.002845550725102 < 4.982098043289534.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 457.83335613905837 (T) = (0 -2014.2610449153567) / Math.Sqrt((11220.52409378932 / (299)) + (83.12061288430901 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.6258339572461665 = (5383.334709078754 - 2014.2610449153567) / 5383.334709078754 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.IO.Tests.Perf_File

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AppendAllText - Duration of single invocation 81.02 μs 70.81 μs 0.87 0.06 False
WriteAllText - Duration of single invocation 120.45 μs 106.15 μs 0.88 0.05 False
AppendAllTextAsync - Duration of single invocation 121.94 μs 109.33 μs 0.90 0.07 False
ReadAllLines - Duration of single invocation 114.45 μs 99.29 μs 0.87 0.02 False
WriteAllText - Duration of single invocation 838.78 μs 744.74 μs 0.89 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_File*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_File.AppendAllText(size: 10000)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 70.8113856923077 < 76.8624665602335.
IsChangePoint: Marked as a change because one of 11/15/2022 10:02:59 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 10.34606719955462 (T) = (0 -71990.74005352563) / Math.Sqrt((5814415.083456031 / (299)) + (2185670.0862522745 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.08851311561348858 = (78981.6521627516 - 71990.74005352563) / 78981.6521627516 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.WriteAllText(size: 10000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 106.15441014921171 < 114.17494549745443.
IsChangePoint: Marked as a change because one of 11/16/2022 6:51:59 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.9829156380811845 (T) = (0 -110077.61566659994) / Math.Sqrt((11958769.307899704 / (299)) + (5309005.029651519 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.07073780296387432 = (118457.00386574598 - 110077.61566659994) / 118457.00386574598 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.AppendAllTextAsync(size: 10000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 109.3302840357143 < 115.0434361953869.
IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 11/15/2022 5:29:43 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 12.59499648950711 (T) = (0 -108335.48816736267) / Math.Sqrt((14310117.69677384 / (299)) + (3579863.1662528752 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.0922357848961241 = (119343.20208355623 - 108335.48816736267) / 119343.20208355623 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.ReadAllLines

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 99.29110133356228 < 107.41684432730067.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/7/2022 4:10:19 PM, 12/16/2022 11:20:37 AM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 40.77537717546181 (T) = (0 -100733.5813316487) / Math.Sqrt((21107773.293588486 / (299)) + (1223439.362033413 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1410980639181624 = (117281.8189130856 - 100733.5813316487) / 117281.8189130856 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_File.WriteAllText(size: 100000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 744.741281547619 < 797.8892787016368.
IsChangePoint: Marked as a change because one of 11/16/2022 11:42:33 AM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 52.655731645349626 (T) = (0 -745361.6775369572) / Math.Sqrt((727251213.2873294 / (299)) + (3924674.4481645874 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.11246192326969019 = (839808.1131154052 - 745361.6775369572) / 839808.1131154052 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnumerateArray - Duration of single invocation 34.93 μs 0.00 ns 0.00 0.14 False
EnumerateArray - Duration of single invocation 36.36 μs 0.00 ns 0.00 0.19 False
EnumerateArray - Duration of single invocation 34.54 μs 0.00 ns 0.00 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfNumbers)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0 < 32.359294512377545.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 438.6403943199348 (T) = (0 -0) / Math.Sqrt((1968389.1351422956 / (299)) + (0 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 1 = (35590.0426730004 - 0) / 35590.0426730004 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: Json400KB)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0 < 34.587668768168726.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 450.5479921785408 (T) = (0 -0) / Math.Sqrt((2098631.2275871392 / (299)) + (0 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 1 = (37746.22548381743 - 0) / 37746.22548381743 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfStrings)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0 < 32.6325175739059.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 455.31891844827567 (T) = (0 -0) / Math.Sqrt((1820823.9767705337 / (299)) + (0 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 1 = (35531.54427118059 - 0) / 35531.54427118059 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Document.Tests.Perf_EnumerateObject

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnumerateProperties - Duration of single invocation 49.21 μs 0.00 ns 0.00 0.15 False
EnumerateProperties - Duration of single invocation 51.02 μs 0.00 ns 0.00 0.16 False
EnumerateProperties - Duration of single invocation 49.03 μs 0.00 ns 0.00 0.17 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateObject*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: StringProperties)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0 < 46.793954779488104.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 528.1031726330704 (T) = (0 -0) / Math.Sqrt((2713148.109716745 / (299)) + (0 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 1 = (50306.01704077589 - 0) / 50306.01704077589 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: ObjectProperties)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0 < 48.51161541863925.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 573.9914817917912 (T) = (0 -0) / Math.Sqrt((2488903.8662649286 / (299)) + (0 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 1 = (52368.95579136021 - 0) / 52368.95579136021 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: NumericProperties)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0 < 47.23158744901034.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 528.6692184707274 (T) = (0 -0) / Math.Sqrt((2705473.0065488024 / (299)) + (0 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 1 = (50288.65646546514 - 0) / 50288.65646546514 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsAllBenchmark - Duration of single invocation 21.43 ns 18.97 ns 0.89 0.23 False
EqualsStaticBenchmark - Duration of single invocation 76.32 ns 67.24 ns 0.88 0.21 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt32>.EqualsAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.973266254850063 < 20.418774901093336.
IsChangePoint: Marked as a change because one of 10/14/2022 1:19:37 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 23.441741439214802 (T) = (0 -18.97908105864672) / Math.Sqrt((2.2435837165662025 / (299)) + (0.056586997693677234 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14489567378961743 = (22.195047407557226 - 18.97908105864672) / 22.195047407557226 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 67.24376402270322 < 72.8475688364945.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/18/2022 12:43:31 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 22.597913473686166 (T) = (0 -67.77780222090256) / Math.Sqrt((17.76535903728671 / (299)) + (0.7927398546644641 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.144729774012596 = (79.24723691001113 - 67.77780222090256) / 79.24723691001113 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Collections.TryGetValueFalse<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableDictionary - Duration of single invocation 183.90 μs 171.37 μs 0.93 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<String, String>.ImmutableDictionary(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 171.37374429347827 < 175.0093400840336.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 5:29:43 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 6.234627784244785 (T) = (0 -173778.65355425997) / Math.Sqrt((13576752.495969964 / (299)) + (16800271.75753597 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.06209557837222509 = (185283.96875734883 - 173778.65355425997) / 185283.96875734883 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 20, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToUtf8Bytes - Duration of single invocation 214.83 μs 190.83 μs 0.89 0.03 False
SerializeObjectProperty - Duration of single invocation 241.21 μs 215.74 μs 0.89 0.03 False
SerializeToString - Duration of single invocation 249.55 μs 222.39 μs 0.89 0.02 False
SerializeToStream - Duration of single invocation 251.65 μs 224.84 μs 0.89 0.03 False
SerializeObjectProperty - Duration of single invocation 257.09 μs 229.54 μs 0.89 0.03 False
SerializeToUtf8Bytes - Duration of single invocation 235.05 μs 207.93 μs 0.88 0.03 False
SerializeToString - Duration of single invocation 227.71 μs 207.12 μs 0.91 0.03 False
SerializeToWriter - Duration of single invocation 211.74 μs 189.40 μs 0.89 0.03 False
SerializeToWriter - Duration of single invocation 231.56 μs 202.55 μs 0.87 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 190.8348770579268 < 204.9669986632624.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 55.080285243325854 (T) = (0 -191148.1910240964) / Math.Sqrt((20381241.026845727 / (112)) + (406894.7764611555 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.12882537883089495 = (219414.32449854657 - 191148.1910240964) / 219414.32449854657 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;.SerializeObjectProperty(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 215.74037721893492 < 228.35246021554482.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 28.503241476636944 (T) = (0 -216329.54331516707) / Math.Sqrt((16744387.673460297 / (112)) + (4106708.0323790703 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.1149060170999592 = (244414.20628163792 - 216329.54331516707) / 244414.20628163792 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;.SerializeToString(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 222.39342754120878 < 237.0191786832966.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 29.89493847482425 (T) = (0 -224527.2314707818) / Math.Sqrt((28116545.68824495 / (112)) + (3737234.932006387 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.11742285881552243 = (254399.55443379286 - 224527.2314707818) / 254399.55443379286 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;.SerializeToStream(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 224.84214559859157 < 239.10243255671395.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 16.716282438074447 (T) = (0 -227089.61797396076) / Math.Sqrt((33789191.55039915 / (112)) + (11163134.665743548 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.10489327293253532 = (253701.1633438935 - 227089.61797396076) / 253701.1633438935 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;.SerializeObjectProperty(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 229.5438022540984 < 243.35029926341053.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 16.815171856470645 (T) = (0 -233851.20913799043) / Math.Sqrt((31344419.162974197 / (112)) + (12701751.327175146 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.10774375331154469 = (262089.74160272043 - 233851.20913799043) / 262089.74160272043 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;.SerializeToUtf8Bytes(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 207.93264353070174 < 221.8217292192497.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 14.074869961649437 (T) = (0 -210753.83753224456) / Math.Sqrt((23959471.9998858 / (112)) + (17176254.23533808 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.11314118307742968 = (237640.79863756377 - 210753.83753224456) / 237640.79863756377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;.SerializeToString(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 207.1245701950188 < 218.52918876509503.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 19.62885538488283 (T) = (0 -207463.7246711032) / Math.Sqrt((19665393.292824354 / (112)) + (8933643.222569205 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.11702658196754831 = (234960.32885497165 - 207463.7246711032) / 234960.32885497165 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;.SerializeToWriter(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 189.39883007028115 < 201.53836247948837.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 43.247340005681124 (T) = (0 -189757.5817462483) / Math.Sqrt((18492690.269205295 / (112)) + (1140104.0536309408 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.12503240868667642 = (216873.84039153127 - 189757.5817462483) / 216873.84039153127 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;.SerializeToWriter(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 202.55295180860807 < 222.28614596674697.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 18.783670456279534 (T) = (0 -207382.9972041389) / Math.Sqrt((25099769.5434832 / (112)) + (11452479.649870161 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.1255904834735675 = (237169.19050465288 - 207382.9972041389) / 237169.19050465288 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<BinaryData>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromString - Duration of single invocation 33.43 μs 31.06 μs 0.93 0.02 False
DeserializeFromString - Duration of single invocation 33.00 μs 31.32 μs 0.95 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;BinaryData&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<BinaryData>.DeserializeFromString(Mode: SourceGen)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.064970813846372 < 31.692687895419986.
IsChangePoint: Marked as a change because one of 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 35.04020709615596 (T) = (0 -31069.528205640265) / Math.Sqrt((339420.42701464146 / (112)) + (8165.01438265104 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.07151006473187288 = (33462.42864406288 - 31069.528205640265) / 33462.42864406288 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;BinaryData&gt;.DeserializeFromString(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.319778486604207 < 31.543471570648062.
IsChangePoint: Marked as a change because one of 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 19.646013319024433 (T) = (0 -31081.74335251432) / Math.Sqrt((296996.1868606996 / (112)) + (43681.056307440915 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.06318935242026172 = (33178.25585438678 - 31081.74335251432) / 33178.25585438678 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Numerics.Tests.Perf_Matrix4x4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultiplyByScalarBenchmark - Duration of single invocation 90.66 ns 68.12 ns 0.75 0.21 False
LerpBenchmark - Duration of single invocation 201.33 ns 124.74 ns 0.62 0.14 False
MultiplyByScalarOperatorBenchmark - Duration of single invocation 85.96 ns 59.51 ns 0.69 0.22 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 68.12061217391397 < 85.77790949525267.
IsChangePoint: Marked as a change because one of 10/5/2022 3:28:28 PM, 10/19/2022 7:27:15 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 34.800488599435 (T) = (0 -69.17571058645146) / Math.Sqrt((99.0788411511684 / (299)) + (1.2393498687183921 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.27687021414450286 = (95.66154228402206 - 69.17571058645146) / 95.66154228402206 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.LerpBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 124.74436967498566 < 194.265812821928.
IsChangePoint: Marked as a change because one of 10/5/2022 3:28:28 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 113.89619641321643 (T) = (0 -125.15044675248859) / Math.Sqrt((116.63825529972794 / (299)) + (0.8203069013336175 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.4038656610327624 = (209.93665114025015 - 125.15044675248859) / 209.93665114025015 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 59.50940750070272 < 81.90911548535715.
IsChangePoint: Marked as a change because one of 10/5/2022 3:28:28 PM, 10/19/2022 7:27:15 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 33.83532718095619 (T) = (0 -60.33445912903617) / Math.Sqrt((77.90382656905014 / (299)) + (3.2512421097029627 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.34861880546242874 = (92.62542369198857 - 60.33445912903617) / 92.62542369198857 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GreaterThanOrEqualAllBenchmark - Duration of single invocation 143.05 ns 130.78 ns 0.91 0.11 False
EqualsAnyBenchmark - Duration of single invocation 134.51 ns 119.02 ns 0.88 0.17 False
EqualsAllBenchmark - Duration of single invocation 30.27 ns 28.34 ns 0.94 0.24 False
GreaterThanOrEqualAnyBenchmark - Duration of single invocation 30.71 ns 26.88 ns 0.88 0.18 False
EqualityOperatorBenchmark - Duration of single invocation 27.88 ns 26.33 ns 0.94 0.28 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.GreaterThanOrEqualAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 130.77866354360057 < 135.81041952163605.
IsChangePoint: Marked as a change because one of 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 24.84527934323502 (T) = (0 -131.22548657513792) / Math.Sqrt((17.51694420151527 / (166)) + (0.46735043169834434 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.07788115992906286 = (142.3086492463845 - 131.22548657513792) / 142.3086492463845 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 119.02499971894083 < 128.32106945834425.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 19.71468719361799 (T) = (0 -117.44342681504375) / Math.Sqrt((26.735977023261093 / (166)) + (4.913061040250803 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.15219724842642984 = (138.52682902604653 - 117.44342681504375) / 138.52682902604653 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.33967856282178 < 28.83052069325937.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 20.056684539709707 (T) = (0 -28.24762255087736) / Math.Sqrt((3.6005301291551564 / (166)) + (0.07761131102255986 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.12046796980863013 = (32.11665019718634 - 28.24762255087736) / 32.11665019718634 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.GreaterThanOrEqualAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.87740192131763 < 27.17082586570145.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 18.18235391943209 (T) = (0 -26.69546064322933) / Math.Sqrt((2.141179938831524 / (166)) + (0.14063059078300327 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.12123024565411214 = (30.37821967723513 - 26.69546064322933) / 30.37821967723513 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.32899923639509 < 26.461881714548312.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 18.802684434455234 (T) = (0 -25.846615336145213) / Math.Sqrt((2.3438179911987183 / (166)) + (0.14037484385557697 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.13000501337505002 = (29.70892445761592 - 25.846615336145213) / 29.70892445761592 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Stack - Duration of single invocation 1.24 ms 383.42 μs 0.31 0.06 False
Queue - Duration of single invocation 1.06 ms 375.58 μs 0.35 0.05 False
List - Duration of single invocation 1.06 ms 376.91 μs 0.36 0.04 False
Array - Duration of single invocation 1.07 ms 387.22 μs 0.36 0.02 False
ICollection - Duration of single invocation 1.06 ms 379.05 μs 0.36 0.04 False
Span - Duration of single invocation 746.46 μs 346.46 μs 0.46 0.11 False
ImmutableArray - Duration of single invocation 1.09 ms 406.68 μs 0.37 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<Int32>.Stack(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 383.4233259146343 < 1.1520970209927093.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 16.975893804360155 (T) = (0 -383420.9783667248) / Math.Sqrt((1383908300086.465 / (299)) + (154686.11798452729 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.7507569451871878 = (1538341.6747747848 - 383420.9783667248) / 1538341.6747747848 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 375.5765333758504 < 1.0062563747713482.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 478.8016032666996 (T) = (0 -375068.9115018528) / Math.Sqrt((650340301.2925456 / (299)) + (108173.09536698285 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.6542223709813323 = (1084711.328972655 - 375068.9115018528) / 1084711.328972655 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 376.91458531504054 < 1.0044148774119546.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 410.23240872048774 (T) = (0 -377906.06000549614) / Math.Sqrt((652353168.2571821 / (299)) + (4050359.4287104853 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.6524951333728081 = (1087484.2233818814 - 377906.06000549614) / 1087484.2233818814 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 387.21881706730767 < 1.0163450028819443.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 530.7074542400034 (T) = (0 -386958.55576574086) / Math.Sqrt((467258062.5364888 / (299)) + (1087001.9405313756 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.6466253696695692 = (1095037.7377230127 - 386958.55576574086) / 1095037.7377230127 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 379.045476686508 < 1.0103705060657049.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 396.7592584239247 (T) = (0 -380662.48194952577) / Math.Sqrt((715130486.1756146 / (299)) + (3903648.82998353 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.6499164329444725 = (1087347.4729225098 - 380662.48194952577) / 1087347.4729225098 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Span(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 346.46237829968936 < 709.8508532316467.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 247.19979768579492 (T) = (0 -346387.1060467336) / Math.Sqrt((785935090.6410106 / (299)) + (312720.2873004134 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.5393218865334999 = (751907.0168978679 - 346387.1060467336) / 751907.0168978679 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 406.68124722222217 < 1.0380656300818452.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 308.95164693092937 (T) = (0 -407054.0914886359) / Math.Sqrt((796542877.6598607 / (299)) + (13210217.372428345 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.6361433812879139 = (1118721.1405675465 - 407054.0914886359) / 1118721.1405675465 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 20, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetBytes - Duration of single invocation 3.56 μs 2.33 μs 0.65 0.11 False
GetBytes - Duration of single invocation 4.21 μs 3.74 μs 0.89 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "ascii")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.3287227090192557 < 2.6615585329768874.
IsChangePoint: Marked as a change because one of 10/5/2022 3:28:28 PM, 10/6/2022 8:29:49 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 46.05328477475609 (T) = (0 -2327.9832891908654) / Math.Sqrt((32782.42856500597 / (299)) + (90.8969231579552 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.18277649796748385 = (2848.649461745703 - 2327.9832891908654) / 2848.649461745703 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.7366565624750017 < 3.9739044179813434.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 44.065991834506356 (T) = (0 -3730.7504909215118) / Math.Sqrt((25769.45156737021 / (299)) + (290.55812799658344 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12425497807390468 = (4260.087579734312 - 3730.7504909215118) / 4260.087579734312 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToStream - Duration of single invocation 135.26 μs 115.13 μs 0.85 0.04 False
SerializeToWriter - Duration of single invocation 123.33 μs 112.04 μs 0.91 0.05 False
SerializeObjectProperty - Duration of single invocation 132.32 μs 115.49 μs 0.87 0.03 False
SerializeToUtf8Bytes - Duration of single invocation 113.83 μs 99.13 μs 0.87 0.02 False
SerializeObjectProperty - Duration of single invocation 144.21 μs 130.46 μs 0.90 0.03 False
SerializeToWriter - Duration of single invocation 112.47 μs 97.76 μs 0.87 0.05 False
SerializeToUtf8Bytes - Duration of single invocation 126.30 μs 110.88 μs 0.88 0.04 False
SerializeToString - Duration of single invocation 132.84 μs 118.90 μs 0.90 0.04 False
SerializeToString - Duration of single invocation 120.03 μs 104.85 μs 0.87 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToStream(Mode: Reflection)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 115.12815464081373 < 123.72423516500106.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 24.22192839697792 (T) = (0 -116610.1281196638) / Math.Sqrt((8858783.246100493 / (112)) + (1738136.3699772938 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.11947751985464944 = (132432.88019224064 - 116610.1281196638) / 132432.88019224064 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;.SerializeToWriter(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 112.03725950255102 < 118.69868154624997.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 17.962586557080186 (T) = (0 -112560.84895241007) / Math.Sqrt((11346635.734130317 / (112)) + (2927243.5488066617 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.1167999006347015 = (127446.59905869646 - 112560.84895241007) / 127446.59905869646 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;.SerializeObjectProperty(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 115.4856871717437 < 123.54509323869786.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 35.79858872829976 (T) = (0 -116693.5917407771) / Math.Sqrt((5199292.236355389 / (112)) + (711957.9871357318 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.11762246213773311 = (132249.05069942088 - 116693.5917407771) / 132249.05069942088 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;.SerializeToUtf8Bytes(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 99.12724146228905 < 107.94338149389577.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 16.034088196484074 (T) = (0 -100785.9383031275) / Math.Sqrt((6800537.7643794315 / (112)) + (3799337.1227390408 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.12596131013878284 = (115310.61436093938 - 100785.9383031275) / 115310.61436093938 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;.SerializeObjectProperty(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 130.4633695764463 < 132.58519211499416.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 19.38988420117855 (T) = (0 -127921.9167204679) / Math.Sqrt((8800561.866107542 / (112)) + (2735438.288328455 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.10705910090546034 = (143259.10802180003 - 127921.9167204679) / 143259.10802180003 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;.SerializeToWriter(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 97.76267523041476 < 107.86404045890076.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 24.817619791057137 (T) = (0 -99296.6066334032) / Math.Sqrt((7810269.481639166 / (112)) + (1598768.0319277532 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.134934186916612 = (114785.03153358518 - 99296.6066334032) / 114785.03153358518 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;.SerializeToUtf8Bytes(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 110.87609592937352 < 120.88873971723915.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 20.23169044003693 (T) = (0 -112069.30805936674) / Math.Sqrt((7120318.22380721 / (112)) + (2764837.1204941045 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.12415243290095479 = (127955.26558412348 - 112069.30805936674) / 127955.26558412348 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;.SerializeToString(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 118.90286458333331 < 126.2274634280423.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 21.28368766282707 (T) = (0 -119887.89998952276) / Math.Sqrt((8431030.518237082 / (112)) + (2547565.9081265675 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.11953223559699916 = (136163.87201956505 - 119887.89998952276) / 136163.87201956505 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;.SerializeToString(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 104.84796212847554 < 114.2397977157343.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 12.174142594046083 (T) = (0 -107256.5982496175) / Math.Sqrt((8564460.394595675 / (112)) + (7781229.250822138 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.1266633939349361 = (122812.43853143475 - 107256.5982496175) / 122812.43853143475 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeObjectProperty - Duration of single invocation 452.97 μs 399.27 μs 0.88 0.04 False
SerializeToStream - Duration of single invocation 581.93 μs 536.79 μs 0.92 0.01 False
SerializeToStream - Duration of single invocation 572.21 μs 531.80 μs 0.93 0.04 False
SerializeToString - Duration of single invocation 588.60 μs 539.78 μs 0.92 0.01 False
SerializeObjectProperty - Duration of single invocation 593.02 μs 543.50 μs 0.92 0.01 False
SerializeToWriter - Duration of single invocation 418.86 μs 377.58 μs 0.90 0.01 False
SerializeToString - Duration of single invocation 469.56 μs 394.10 μs 0.84 0.03 False
SerializeToUtf8Bytes - Duration of single invocation 558.73 μs 514.08 μs 0.92 0.01 False
SerializeToUtf8Bytes - Duration of single invocation 410.75 μs 364.57 μs 0.89 0.01 False
SerializeToWriter - Duration of single invocation 575.13 μs 538.34 μs 0.94 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeObjectProperty(Mode: SourceGen)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 399.2703885416667 < 425.74459542049317.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 12.0774600558861 (T) = (0 -406760.55249458033) / Math.Sqrt((102405122.64523561 / (112)) + (86746286.57911086 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.11260250211744217 = (458374.68943192 - 406760.55249458033) / 458374.68943192 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;.SerializeToStream(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 536.7858581896551 < 553.0616246403769.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 44.9593500016396 (T) = (0 -538970.7539410183) / Math.Sqrt((79225511.37615985 / (112)) + (3334163.651762695 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.08907632998654513 = (591674.9906532315 - 538970.7539410183) / 591674.9906532315 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;.SerializeToStream(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 531.8011988916256 < 539.3675264393188.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 25.299361663691244 (T) = (0 -530902.1604710591) / Math.Sqrt((156059833.94421276 / (112)) + (13973415.463688223 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.08885647148614637 = (582676.7615163794 - 530902.1604710591) / 582676.7615163794 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;.SerializeToString(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 539.7809627083334 < 556.7941086983617.
IsChangePoint: Marked as a change because one of 12/3/2022 7:24:45 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 24.676123886400543 (T) = (0 -544104.8356156067) / Math.Sqrt((69377460.33190408 / (112)) + (15964438.178681718 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.08134691622376156 = (592285.4287703425 - 544104.8356156067) / 592285.4287703425 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;.SerializeObjectProperty(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 543.4974365433674 < 561.3487592633928.
IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 17.358733705279633 (T) = (0 -547628.6670630279) / Math.Sqrt((105352868.4741588 / (112)) + (40468671.87874599 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.08698768967353515 = (599804.2533152843 - 547628.6670630279) / 599804.2533152843 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;.SerializeToWriter(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 377.5807050709706 < 401.9083108657095.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 22.93088558855286 (T) = (0 -381678.5386944185) / Math.Sqrt((71333532.45641303 / (112)) + (23040659.954705637 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.12095125680249512 = (434194.96546468855 - 381678.5386944185) / 434194.96546468855 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;.SerializeToString(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 394.1041199776786 < 415.0985528956886.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 33.46433135305519 (T) = (0 -393916.04928273207) / Math.Sqrt((90380201.92359394 / (112)) + (9185489.112178886 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.1213722409505173 = (448330.985705344 - 393916.04928273207) / 448330.985705344 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;.SerializeToUtf8Bytes(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 514.0822866987179 < 530.4823953082074.
IsChangePoint: Marked as a change because one of 12/3/2022 7:24:45 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 22.2995992753501 (T) = (0 -516460.7024142247) / Math.Sqrt((75612417.57539576 / (112)) + (21037211.659890577 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.08709786184234371 = (565735.0123601452 - 516460.7024142247) / 565735.0123601452 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;.SerializeToUtf8Bytes(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 364.5709711378737 < 389.5396598882205.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 39.35846903184889 (T) = (0 -366564.04904080415) / Math.Sqrt((60968634.064684644 / (112)) + (7095717.779835562 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.13077813150160725 = (421715.17114963377 - 366564.04904080415) / 421715.17114963377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;.SerializeToWriter(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 538.3383489839902 < 542.9956828127953.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 23.298524527011647 (T) = (0 -532745.942126642) / Math.Sqrt((94060916.43976454 / (112)) + (18524988.168398153 / (5))) is greater than 1.980807541103565 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (112) + (5) - 2, .975) and 0.08528142637569575 = (582415.1356365186 - 532745.942126642) / 582415.1356365186 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Dec 20, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf16 - Duration of single invocation 10.70 μs 8.88 μs 0.83 0.03 False
EncodeUtf16 - Duration of single invocation 9.89 μs 8.21 μs 0.83 0.06 False
EncodeUtf16 - Duration of single invocation 9.93 μs 8.51 μs 0.86 0.05 False
EncodeUtf16 - Duration of single invocation 10.37 μs 8.69 μs 0.84 0.03 False
EncodeUtf16 - Duration of single invocation 10.04 μs 8.41 μs 0.84 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,&lorem ipsum=dolor sit amet,512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.876255451847499 < 10.168981236834732.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 16.066990254968328 (T) = (0 -9095.573588142226) / Math.Sqrt((214145.9531667033 / (299)) + (106503.48823490407 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.20767564838484495 = (11479.60878597367 - 9095.573588142226) / 11479.60878597367 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,no &lt;escaping /&gt; required,512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.21130405470671 < 9.508859574579226.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 79.64255717211945 (T) = (0 -8224.592391692251) / Math.Sqrt((288831.96569551295 / (299)) + (297.05674619962167 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.2366897811162304 = (10774.901459749253 - 8224.592391692251) / 10774.901459749253 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,hello "there",512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.514601750896672 < 9.43636646680062.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 41.25463076817599 (T) = (0 -8323.079956170279) / Math.Sqrt((296209.1846426404 / (299)) + (13308.43284271824 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.23050518249077395 = (10816.291113059364 - 8323.079956170279) / 10816.291113059364 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,&Hello+&lt;World&gt;!,512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.689240060715608 < 9.861213444080576.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 87.07914780676596 (T) = (0 -8692.954261648849) / Math.Sqrt((241266.63517020727 / (299)) + (202.30279273843945 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.22576504566842978 = (11227.79876188081 - 8692.954261648849) / 11227.79876188081 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,�2020,512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.406419744047872 < 9.48389403398682.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 52.20322640088459 (T) = (0 -8445.409353410625) / Math.Sqrt((274794.756746676 / (299)) + (6095.551858828257 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.22228766358587934 = (10859.29714365951 - 8445.409353410625) / 10859.29714365951 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOfValue - Duration of single invocation 220.58 ns 72.22 ns 0.33 0.06 False
LastIndexOfAnyValues - Duration of single invocation 3.70 μs 3.14 μs 0.85 0.11 False
LastIndexOfValue - Duration of single invocation 88.80 ns 34.51 ns 0.39 0.08 False
LastIndexOfValue - Duration of single invocation 2.37 μs 686.52 ns 0.29 0.05 False
IndexOfAnyTwoValues - Duration of single invocation 318.54 ns 267.88 ns 0.84 0.05 False
IndexOfAnyTwoValues - Duration of single invocation 3.79 μs 3.19 μs 0.84 0.08 False
IndexOfValue - Duration of single invocation 51.03 ns 34.82 ns 0.68 0.22 False
IndexOfAnyThreeValues - Duration of single invocation 4.94 μs 4.12 μs 0.83 0.10 False
Fill - Duration of single invocation 134.44 ns 117.95 ns 0.88 0.27 False
IndexOfValue - Duration of single invocation 2.05 μs 686.11 ns 0.33 0.08 False
Fill - Duration of single invocation 1.68 μs 1.32 μs 0.79 0.17 False
LastIndexOfAnyValues - Duration of single invocation 317.33 ns 277.43 ns 0.87 0.05 False
EndsWith - Duration of single invocation 83.34 ns 74.40 ns 0.89 0.33 False
IndexOfValue - Duration of single invocation 157.38 ns 75.02 ns 0.48 0.08 False
IndexOfAnyThreeValues - Duration of single invocation 387.89 ns 336.43 ns 0.87 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.LastIndexOfValue(Size: 33)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 72.22469628539945 < 209.76738579395584.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 9.394631653842918 (T) = (0 -73.13724543565675) / Math.Sqrt((82415.55365570127 / (70)) + (1.5525658616641886 / (5))) is greater than 1.9929971258892505 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (5) - 2, .975) and 0.8150930619847956 = (395.53543106988695 - 73.13724543565675) / 395.53543106988695 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.LastIndexOfAnyValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.1404835462962963 < 3.516855589548645.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/1/2022 8:17:56 AM, 12/1/2022 7:29:59 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 49.040538382099996 (T) = (0 -3179.391438694383) / Math.Sqrt((21601.38129005949 / (299)) + (590.0053997903883 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.17542774707877348 = (3855.8069683168424 - 3179.391438694383) / 3855.8069683168424 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.LastIndexOfValue(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.50624723866356 < 84.91407459507947.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 17.89197397845549 (T) = (0 -33.93721660833381) / Math.Sqrt((1221.578200561554 / (70)) + (0.33253248318697815 / (5))) is greater than 1.9929971258892505 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (5) - 2, .975) and 0.68814123769546 = (108.82239241106538 - 33.93721660833381) / 108.82239241106538 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.LastIndexOfValue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 686.5245368601984 < 2.230705136178047.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 16.881139293005244 (T) = (0 -689.9103812299549) / Math.Sqrt((5365970.47894813 / (299)) + (69.04341542654582 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.7663101122377507 = (2952.2474756453903 - 689.9103812299549) / 2952.2474756453903 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfAnyTwoValues(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 267.8759168290056 < 300.8358657517354.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 31.620691193074247 (T) = (0 -271.6981648066295) / Math.Sqrt((27.984672300800316 / (70)) + (9.551470386501723 / (5))) is greater than 1.9929971258892505 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (5) - 2, .975) and 0.15030127682849206 = (319.75823594569346 - 271.6981648066295) / 319.75823594569346 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfAnyTwoValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.192799394250513 < 3.606620624836777.
IsChangePoint: Marked as a change because one of 11/18/2022 1:00:00 AM, 12/2/2022 5:21:30 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 44.35561794411078 (T) = (0 -3210.0308317906506) / Math.Sqrt((20621.340107172036 / (299)) + (1003.4342505886148 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.18494070378919483 = (3938.401594477876 - 3210.0308317906506) / 3938.401594477876 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfValue(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.81516882547416 < 48.83478467204152.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 44.77854176450618 (T) = (0 -35.27035633253126) / Math.Sqrt((7.950889932344679 / (70)) + (0.15624691942419358 / (5))) is greater than 1.9929971258892505 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (5) - 2, .975) and 0.32576576967112725 = (52.31172602335446 - 35.27035633253126) / 52.31172602335446 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfAnyThreeValues(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.120141209106985 < 4.698165624952159.
IsChangePoint: Marked as a change because one of 10/11/2022 1:45:10 PM, 10/12/2022 11:17:15 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 48.56171504802444 (T) = (0 -4127.528618263817) / Math.Sqrt((49420.70522562229 / (299)) + (1279.006586327563 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.19447701165405862 = (5124.035785420939 - 4127.528618263817) / 5124.035785420939 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.Fill(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 117.95221841549801 < 132.28340221749664.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 22.929279244163787 (T) = (0 -117.98927092225881) / Math.Sqrt((44.90680931046178 / (70)) + (1.6294797275978716 / (5))) is greater than 1.9929971258892505 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (5) - 2, .975) and 0.1604694374068215 = (140.5419602090583 - 117.98927092225881) / 140.5419602090583 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 686.111005450699 < 1.9453061074473.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 432.89245604600166 (T) = (0 -688.7760905228109) / Math.Sqrt((2469.381289763611 / (299)) + (12.192597201223883 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.672732455736633 = (2104.6269408509443 - 688.7760905228109) / 2104.6269408509443 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.Fill(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3221070211827153 < 1.5997392072511445.
IsChangePoint: Marked as a change because one of 10/1/2022 3:26:37 AM, 10/2/2022 4:08:45 AM, 10/2/2022 11:57:04 PM, 10/3/2022 8:26:24 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 53.06581826449089 (T) = (0 -1321.7725771217213) / Math.Sqrt((17827.497587562273 / (299)) + (3.69563198571578 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.2377583268987308 = (1734.0597132979299 - 1321.7725771217213) / 1734.0597132979299 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.LastIndexOfAnyValues(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 277.42950202411276 < 295.7233027075381.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 34.54283747809054 (T) = (0 -275.4216191268873) / Math.Sqrt((42.1859312519928 / (70)) + (4.991809388256781 / (5))) is greater than 1.9929971258892505 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (5) - 2, .975) and 0.13695871323640577 = (319.1291347830168 - 275.4216191268873) / 319.1291347830168 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.EndsWith(Size: 4)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 74.39712852243528 < 76.64102731092552.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 7.164153160970241 (T) = (0 -74.3890362231521) / Math.Sqrt((15.309260960362655 / (69)) + (3.999296906775429 / (5))) is greater than 1.9934635666612894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (69) + (5) - 2, .975) and 0.08871160228872205 = (81.63061925289722 - 74.3890362231521) / 81.63061925289722 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfValue(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 75.020689093126 < 149.53908415600077.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 100.20119917450805 (T) = (0 -72.36675435445576) / Math.Sqrt((23.42041266375031 / (70)) + (2.2222210024522653 / (5))) is greater than 1.9929971258892505 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (5) - 2, .975) and 0.5499762793781269 = (160.80653316330637 - 72.36675435445576) / 160.80653316330637 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfAnyThreeValues(Size: 33)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 336.43303157448617 < 374.4529823406164.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 34.92318099875154 (T) = (0 -333.4550394720735) / Math.Sqrt((92.70501289030433 / (70)) + (9.282673301100271 / (5))) is greater than 1.9929971258892505 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (5) - 2, .975) and 0.15739013564969065 = (395.74072602292944 - 333.4550394720735) / 395.74072602292944 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Int

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsAllBenchmark - Duration of single invocation 33.38 ns 28.65 ns 0.86 0.23 False
EqualsAnyBenchmark - Duration of single invocation 87.87 ns 82.49 ns 0.94 0.10 False
EqualityOperatorBenchmark - Duration of single invocation 27.81 ns 25.59 ns 0.92 0.28 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualsAllBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.65415333322152 < 29.883392744953973.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 13.01900743802451 (T) = (0 -28.346127115840478) / Math.Sqrt((3.3837311663382983 / (166)) + (0.24402844399567056 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.10778849539304078 = (31.770636188251835 - 28.346127115840478) / 31.770636188251835 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualsAnyBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 82.49115423364742 < 84.58375088291317.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 22.730212889244378 (T) = (0 -82.38672405978113) / Math.Sqrt((6.953810626925045 / (166)) + (0.45593432303611664 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.0914429916395779 = (90.67865120368819 - 82.38672405978113) / 90.67865120368819 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualityOperatorBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.592295904992664 < 26.404169767442646.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 16.82434916344562 (T) = (0 -25.70507777874473) / Math.Sqrt((4.215618712774697 / (166)) + (0.14135781403574155 / (5))) is greater than 1.9741004473987562 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (166) + (5) - 2, .975) and 0.13166220727610625 = (29.6026246860802 - 25.70507777874473) / 29.6026246860802 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsStaticBenchmark - Duration of single invocation 148.55 ns 132.81 ns 0.89 0.12 False
EqualsAllBenchmark - Duration of single invocation 23.95 ns 18.65 ns 0.78 0.29 False
SquareRootBenchmark - Duration of single invocation 127.01 ns 111.73 ns 0.88 0.09 False
DotBenchmark - Duration of single invocation 123.53 ns 101.40 ns 0.82 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt16>.EqualsStaticBenchmark

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 132.8108106566714 < 142.08343633371305.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 10:51:47 AM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 44.22775665795556 (T) = (0 -132.78980885305373) / Math.Sqrt((33.07909364088113 / (299)) + (0.31860295576599573 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12209365128120919 = (151.2573739178969 - 132.78980885305373) / 151.2573739178969 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.EqualsAllBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.654264688130986 < 19.91427731310107.
IsChangePoint: Marked as a change because one of 11/15/2022 5:29:43 PM, 12/1/2022 11:48:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 37.15111583684087 (T) = (0 -18.715231582717657) / Math.Sqrt((2.711287012800812 / (299)) + (0.006455387620711265 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.16807986167209288 = (22.496428107070198 - 18.715231582717657) / 22.496428107070198 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.SquareRootBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 111.72910282008883 < 119.38240221305891.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 41.90591004050843 (T) = (0 -111.74860106531612) / Math.Sqrt((36.063974751502464 / (299)) + (0.3011757079916471 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1375407949273073 = (129.56972388728502 - 111.74860106531612) / 129.56972388728502 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.DotBenchmark

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 101.39834846058992 < 112.30759917868815.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/17/2022 10:44:50 PM, 12/20/2022 4:30:45 AM falls between 12/11/2022 2:11:02 PM and 12/20/2022 4:30:45 AM.
IsImprovementStdDev: Marked as improvement because 44.59688614200897 (T) = (0 -100.89022902953313) / Math.Sqrt((43.7920239913741 / (299)) + (0.4748359501569701 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.17843900279195202 = (122.80308993780555 - 100.89022902953313) / 122.80308993780555 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@lewing
Copy link
Member

lewing commented Dec 20, 2022

this is dotnet/runtime#79215 cc @BrzVlad @SamMonoRT

@kotlarmilos kotlarmilos mentioned this issue Apr 4, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-improvement runtime-mono untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lewing @kotlarmilos