Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 158 Improvements on 2/15/2023 8:55:14 PM #13314

Open
performanceautofiler bot opened this issue Feb 21, 2023 · 9 comments
Open

[Perf] Linux/x64: 158 Improvements on 2/15/2023 8:55:14 PM #13314

performanceautofiler bot opened this issue Feb 21, 2023 · 9 comments
Labels
arch-wasm arch-x64 branch-refs/heads/main kind-micro os-linux perf-improvement untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Feb 21, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString_NonFlags_Small - Duration of single invocation 953.09 ns 720.20 ns 0.76 0.11 False
ToString_Format_Flags_Large - Duration of single invocation 817.08 ns 582.00 ns 0.71 0.14 False
StringFormat - Duration of single invocation 8.22 μs 6.93 μs 0.84 0.11 False
InterpolateIntoStringBuilder_NonFlags - Duration of single invocation 13.15 μs 11.27 μs 0.86 0.07 False
InterpolateIntoString - Duration of single invocation 13.77 μs 12.01 μs 0.87 0.05 False
ToString_Flags - Duration of single invocation 1.09 μs 827.19 ns 0.76 0.17 False
InterpolateIntoStringBuilder_Flags - Duration of single invocation 12.67 μs 11.35 μs 0.90 0.06 False
ToString_NonFlags_Large - Duration of single invocation 859.71 ns 619.81 ns 0.72 0.16 False
ToString_Flags - Duration of single invocation 1.12 μs 847.41 ns 0.75 0.11 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.ToString_NonFlags_Small(value: -1)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 720.203303281232 < 936.6130330016003.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/15/2023 7:15:22 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 35.249899840322435 (T) = (0 -734.5593417431369) / Math.Sqrt((7771.825305781427 / (194)) + (473.89661609312645 / (32))) is greater than 1.9706109611022053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (194) + (32) - 2, .975) and 0.26224692196468125 = (995.670995638965 - 734.5593417431369) / 995.670995638965 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.ToString_Format_Flags_Large(value: All, format: "d")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 582.0009367748637 < 754.8838858255058.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 23.798933708350674 (T) = (0 -596.3424029089945) / Math.Sqrt((12773.723670705198 / (194)) + (252.0049371920753 / (32))) is greater than 1.9706109611022053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (194) + (32) - 2, .975) and 0.25520472290620283 = (800.6796246559617 - 596.3424029089945) / 800.6796246559617 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.StringFormat(value: 32)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.929294960346256 < 7.901238860994894.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 26.096534015279584 (T) = (0 -6948.659460474508) / Math.Sqrt((1311801.5030933253 / (194)) + (55391.23441282889 / (32))) is greater than 1.9706109611022053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (194) + (32) - 2, .975) and 0.2571160714029868 = (9353.627387790611 - 6948.659460474508) / 9353.627387790611 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.InterpolateIntoStringBuilder_NonFlags(value: 42)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.268638191893219 < 12.179635376252927.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 7.777918305092729 (T) = (0 -11470.650397209016) / Math.Sqrt((1898370.6616823785 / (193)) + (37997.92541861701 / (32))) is greater than 1.9706589608356722 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (193) + (32) - 2, .975) and 0.06646124532748722 = (12287.278208640564 - 11470.650397209016) / 12287.278208640564 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.InterpolateIntoString(value: 32)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.010418258212374 < 12.842441585561138.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/15/2023 10:25:39 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 8.831736644254791 (T) = (0 -12181.7619402098) / Math.Sqrt((1738702.7511535508 / (195)) + (46435.448521687074 / (32))) is greater than 1.9705633902925863 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (195) + (32) - 2, .975) and 0.06874517664835579 = (13081.018894878718 - 12181.7619402098) / 13081.018894878718 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.ToString_Flags(value: 32)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 827.1881842248863 < 1.0485795461674177.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 34.53372202962532 (T) = (0 -815.52673279313) / Math.Sqrt((7377.452199015507 / (195)) + (590.6052572897656 / (32))) is greater than 1.9705633902925863 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (195) + (32) - 2, .975) and 0.24110262676917632 = (1074.6205765889272 - 815.52673279313) / 1074.6205765889272 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.InterpolateIntoStringBuilder_Flags(value: 32)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.345271415678186 < 11.822569956623054.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 8.342950605926932 (T) = (0 -11237.086838265774) / Math.Sqrt((1800350.502344818 / (192)) + (39521.06018303514 / (32))) is greater than 1.9707073953201992 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (192) + (32) - 2, .975) and 0.07104846821319175 = (12096.526518075276 - 11237.086838265774) / 12096.526518075276 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.ToString_NonFlags_Large(value: 42)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 619.8104680394317 < 822.6725513279544.
IsChangePoint: Marked as a change because one of 1/9/2023 3:58:12 PM, 1/12/2023 3:42:01 AM, 1/15/2023 4:08:29 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 23.839697624176495 (T) = (0 -641.3830462586478) / Math.Sqrt((20089.012642210015 / (194)) + (457.10674993000526 / (32))) is greater than 1.9706109611022053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (194) + (32) - 2, .975) and 0.2874856831883279 = (900.1686438087036 - 641.3830462586478) / 900.1686438087036 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.ToString_Flags(value: 36)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 847.4135177508504 < 1.0946999415456142.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 31.071613780238597 (T) = (0 -842.616156693363) / Math.Sqrt((7068.7545966472435 / (194)) + (1170.6915718188015 / (32))) is greater than 1.9706109611022053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (194) + (32) - 2, .975) and 0.2396055306900975 = (1108.1303069682779 - 842.616156693363) / 1108.1303069682779 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryFormat4 - Duration of single invocation 1.88 μs 992.86 ns 0.53 0.11 False
ToString3 - Duration of single invocation 1.70 μs 1.05 μs 0.62 0.06 False
ToStringL - Duration of single invocation 2.45 μs 1.76 μs 0.72 0.09 False
ToString4 - Duration of single invocation 2.13 μs 1.28 μs 0.60 0.08 False
TryFormat2 - Duration of single invocation 917.72 ns 518.13 ns 0.56 0.09 False
TryFormat3 - Duration of single invocation 1.40 μs 729.79 ns 0.52 0.09 False
TryFormatL - Duration of single invocation 2.18 μs 1.49 μs 0.68 0.09 False
ToString2 - Duration of single invocation 1.24 μs 806.70 ns 0.65 0.09 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Version*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Version.TryFormat4

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 992.860077158324 < 1.741770321024235.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 33.914861440937564 (T) = (0 -991.9501992425254) / Math.Sqrt((121261.67898038065 / (299)) + (1160.6088195635218 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4181527934196376 = (1704.8293573022788 - 991.9501992425254) / 1704.8293573022788 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.ToString3

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0522938944031506 < 1.6112435431371757.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 42.02507418705528 (T) = (0 -1038.9232369445745) / Math.Sqrt((51295.002386310916 / (299)) + (1185.5122453880688 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.36877899324358693 = (1645.894584977735 - 1038.9232369445745) / 1645.894584977735 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.ToStringL

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.759734170347981 < 2.3714869509610077.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 58.630484397544656 (T) = (0 -1752.3469947296305) / Math.Sqrt((87500.96496173713 / (299)) + (2645.621073079174 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.39327520460843274 = (2888.207319100422 - 1752.3469947296305) / 2888.207319100422 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.ToString4

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2773342048199834 < 2.0173746366166463.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 38.84384159377969 (T) = (0 -1304.311543290916) / Math.Sqrt((96260.83418067398 / (299)) + (1897.6849864039011 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3676853351333497 = (2062.757066635461 - 1304.311543290916) / 2062.757066635461 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.TryFormat2

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 518.1311255992244 < 880.1244498680181.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 34.2167451024951 (T) = (0 -525.0333459397746) / Math.Sqrt((30015.323269150696 / (299)) + (204.56373606478658 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4024254445978987 = (878.6072653084859 - 525.0333459397746) / 878.6072653084859 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.TryFormat3

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 729.7945119526396 < 1.3210621384923098.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 34.62216904906278 (T) = (0 -751.6824621483197) / Math.Sqrt((67212.88168789464 / (299)) + (610.0615703044 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.41835511773954637 = (1292.3391661713706 - 751.6824621483197) / 1292.3391661713706 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.TryFormatL

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.486235989131342 < 2.037452798548641.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 63.55858470161165 (T) = (0 -1410.362470835293) / Math.Sqrt((67980.23808052542 / (299)) + (1296.642703539459 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.42448888026706155 = (2450.625926202366 - 1410.362470835293) / 2450.625926202366 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.ToString2

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 806.6976407488386 < 1.183086656598896.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 42.03770357063851 (T) = (0 -806.8239625525238) / Math.Sqrt((19876.602651253037 / (299)) + (1244.9980230023202 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3484777079102295 = (1238.367393331424 - 806.8239625525238) / 1238.367393331424 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Tests.Perf_Decimal

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Round - Duration of single invocation 374.85 ns 168.29 ns 0.45 0.06 False
Floor - Duration of single invocation 312.27 ns 108.96 ns 0.35 0.05 False
Divide - Duration of single invocation 1.71 μs 831.15 ns 0.49 0.06 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Decimal.Round

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 168.28632164937076 < 355.2228549720679.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 25.268794834230206 (T) = (0 -169.24566418931315) / Math.Sqrt((9620.159988871917 / (299)) + (3.38828547156693 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.45896771157944555 = (312.81989598697544 - 169.24566418931315) / 312.81989598697544 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Decimal.Floor

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 108.95687509549275 < 298.1062596391903.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 28.76282852439328 (T) = (0 -110.12381189581542) / Math.Sqrt((8924.69071801176 / (299)) + (4.331499159417129 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.5885096095285645 = (267.62183138626636 - 110.12381189581542) / 267.62183138626636 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Decimal.Divide

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 831.1528819482463 < 1.6344916751246217.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 31.59828232508761 (T) = (0 -846.4472227867492) / Math.Sqrt((143824.18006140908 / (299)) + (219.2482947279804 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4519185862504692 = (1544.382278895466 - 846.4472227867492) / 1544.382278895466 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Feb 21, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Tests.Perf_Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringWithFormat - Duration of single invocation 4.69 μs 3.58 μs 0.76 0.03 False
TryParse - Duration of single invocation 1.85 μs 1.47 μs 0.79 0.11 False
ToStringWithFormat - Duration of single invocation 3.66 μs 2.60 μs 0.71 0.08 False
Parse - Duration of single invocation 1.95 μs 1.42 μs 0.73 0.16 False
ToStringWithFormat - Duration of single invocation 3.61 μs 2.71 μs 0.75 0.09 False
ToString - Duration of single invocation 4.74 μs 3.39 μs 0.71 0.04 False
ToStringWithFormat - Duration of single invocation 4.80 μs 3.23 μs 0.67 0.06 False
TryParse - Duration of single invocation 1.87 μs 1.58 μs 0.85 0.12 False
ToString - Duration of single invocation 4.48 μs 3.22 μs 0.72 0.05 False
ToStringWithFormat - Duration of single invocation 4.67 μs 3.23 μs 0.69 0.06 False
Parse - Duration of single invocation 1.89 μs 1.53 μs 0.81 0.16 False
ToStringWithFormat - Duration of single invocation 3.93 μs 2.58 μs 0.66 0.09 False
ToStringWithCultureInfo - Duration of single invocation 4.64 μs 3.17 μs 0.68 0.04 False
ToStringWithFormat - Duration of single invocation 3.86 μs 2.65 μs 0.69 0.10 False
ToStringWithCultureInfo - Duration of single invocation 3.54 μs 2.54 μs 0.72 0.09 False
ToStringWithFormat - Duration of single invocation 4.62 μs 3.29 μs 0.71 0.04 False
ToStringWithFormat - Duration of single invocation 4.46 μs 3.23 μs 0.73 0.06 False
ToStringWithCultureInfo - Duration of single invocation 4.57 μs 3.30 μs 0.72 0.04 False
ToString - Duration of single invocation 3.62 μs 2.75 μs 0.76 0.08 False
ToStringWithFormat - Duration of single invocation 4.75 μs 3.20 μs 0.67 0.06 False
ToStringWithFormat - Duration of single invocation 3.88 μs 2.59 μs 0.67 0.08 False
ToStringWithFormat - Duration of single invocation 4.69 μs 3.30 μs 0.70 0.05 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G17")

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.5774711135024035 < 4.495157786187887.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 55.113969632460304 (T) = (0 -3478.039213438655) / Math.Sqrt((575187.2133004548 / (299)) + (4645.165828407662 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.41886287050699683 = (5984.885557859594 - 3478.039213438655) / 5984.885557859594 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.TryParse(value: "3.4028235E+38")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.4690245411550487 < 1.7579018589235647.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 49.02293288752287 (T) = (0 -1468.3737432885086) / Math.Sqrt((51772.207085444395 / (299)) + (2327.2840064858656 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.34362007572123104 = (2237.078997962894 - 1468.3737432885086) / 2237.078997962894 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "R")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.5968723648823793 < 3.444335029886189.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 64.75103347320517 (T) = (0 -2632.298437616153) / Math.Sqrt((144246.67310492072 / (299)) + (4822.136971859572 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3823155660692258 = (4261.558642274548 - 2632.298437616153) / 4261.558642274548 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.Parse(value: "3.4028235E+38")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.4163135114434553 < 1.7526706850376936.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 50.55616935220928 (T) = (0 -1468.2322058332904) / Math.Sqrt((49567.52052960134 / (299)) + (2132.865303386947 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.34424458187046214 = (2238.9936327499104 - 1468.2322058332904) / 2238.9936327499104 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.7095136785651324 < 3.464840995392414.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 66.66477497872283 (T) = (0 -2596.635550813353) / Math.Sqrt((137265.5532614135 / (299)) + (3921.4487770273827 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.3831986795288039 = (4209.841102204016 - 2596.635550813353) / 4209.841102204016 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToString(value: -3.4028235E+38)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.385403983216237 < 4.441041149091492.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 58.922750488064935 (T) = (0 -3272.0308516317164) / Math.Sqrt((485181.97399748943 / (299)) + (6074.9244927000245 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4339627575969285 = (5780.592877140978 - 3272.0308516317164) / 5780.592877140978 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "R")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.232015682544646 < 4.473905890951893.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 55.986778382280306 (T) = (0 -3332.2572303242705) / Math.Sqrt((568639.7059280674 / (299)) + (7724.818770889037 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4375132304157273 = (5924.152194347793 - 3332.2572303242705) / 5924.152194347793 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.TryParse(value: "-3.4028235E+38")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5821390541640044 < 1.782588070228082.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 47.03837153698736 (T) = (0 -1516.4317900160358) / Math.Sqrt((60938.10679728252 / (299)) + (2166.637231402078 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.33824045949459525 = (2291.5148134591204 - 1516.4317900160358) / 2291.5148134591204 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToString(value: 3.4028235E+38)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.221874745327666 < 4.345334607087283.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 59.591256075610474 (T) = (0 -3247.7170597282066) / Math.Sqrt((476696.35734723456 / (299)) + (4633.353579082241 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.4338162177633749 = (5736.153456919204 - 3247.7170597282066) / 5736.153456919204 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "R")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.2339588653897313 < 4.473476448059912.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 57.379883688417515 (T) = (0 -3311.1787908386136) / Math.Sqrt((598730.84388463 / (299)) + (3733.1263085518012 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.44374140404017215 = (5952.588984490483 - 3311.1787908386136) / 5952.588984490483 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.Parse(value: "-3.4028235E+38")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5290490803785541 < 1.8110129333500076.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 48.195662668695114 (T) = (0 -1511.7586871163844) / Math.Sqrt((57727.464588161485 / (299)) + (2222.347513514969 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.34060431080513265 = (2292.6426603763 - 1511.7586871163844) / 2292.6426603763 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "E")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.5841209351424728 < 3.745859149463629.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/13/2023 11:01:49 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 69.1812357914068 (T) = (0 -2639.4450891954575) / Math.Sqrt((164364.165728483 / (299)) + (4618.0569910946615 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.40845917492164724 = (4461.982972765825 - 2639.4450891954575) / 4461.982972765825 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithCultureInfo(value: -3.4028235E+38, culture: zh)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.169309015208369 < 4.411603992318676.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 59.59710752871172 (T) = (0 -3282.0183422046825) / Math.Sqrt((473810.25992278307 / (299)) + (4806.716605190446 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4306337504358243 = (5764.33595197629 - 3282.0183422046825) / 5764.33595197629 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "E")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.6501768867924524 < 3.6502022113372847.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/13/2023 11:01:49 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 70.2719631173994 (T) = (0 -2586.5091385303394) / Math.Sqrt((168995.91930595346 / (299)) + (3209.9259839572087 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.41207070726597245 = (4399.35408984707 - 2586.5091385303394) / 4399.35408984707 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithCultureInfo(value: 12345, culture: zh)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.539613591743523 < 3.4810931615236513.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/13/2023 11:01:49 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 70.17843078982756 (T) = (0 -2556.1194853669112) / Math.Sqrt((114657.89980376129 / (299)) + (3443.2414212098033 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3782674262649675 = (4111.284486851204 - 2556.1194853669112) / 4111.284486851204 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.293693477177242 < 4.408844477839262.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 57.36494553361398 (T) = (0 -3266.385819907542) / Math.Sqrt((559290.066995496 / (299)) + (5151.506086205469 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4418305309295358 = (5851.9607411475745 - 3266.385819907542) / 5851.9607411475745 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G17")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.234194810188764 < 4.271908692107936.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 61.826405657961494 (T) = (0 -3251.3646266869214) / Math.Sqrt((338046.9113292578 / (299)) + (5523.093764110315 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.407040125687926 = (5483.279337339667 - 3251.3646266869214) / 5483.279337339667 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithCultureInfo(value: 3.4028235E+38, culture: zh)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.2970268380784815 < 4.337494505748633.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 60.67755768468015 (T) = (0 -3239.921089946025) / Math.Sqrt((460081.4812106565 / (299)) + (3860.0969926717144 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4327509427265213 = (5711.637680843273 - 3239.921089946025) / 5711.637680843273 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToString(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.7498159857077393 < 3.4122851145382302.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/13/2023 11:01:49 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 68.36390753235068 (T) = (0 -2568.26367010364) / Math.Sqrt((114028.48650928604 / (299)) + (4112.5371732423355 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3754163220335049 = (4111.960912051581 - 2568.26367010364) / 4111.960912051581 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G17")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.197924402409674 < 4.255801518093886.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 60.56588836139471 (T) = (0 -3226.9582947506087) / Math.Sqrt((339953.351223029 / (299)) + (5600.175676931258 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.40469994583563956 = (5420.725686444597 - 3226.9582947506087) / 5420.725686444597 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "E")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.587297105808135 < 3.7188510260793626.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/13/2023 11:01:49 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 71.91439646005708 (T) = (0 -2548.4546547244577) / Math.Sqrt((172240.9692196771 / (299)) + (3090.8275564948503 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.42258832049225653 = (4413.583488468874 - 2548.4546547244577) / 4413.583488468874 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.298862769093061 < 4.448847084275705.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 57.565378962530595 (T) = (0 -3316.7814487305236) / Math.Sqrt((570807.4409583675 / (299)) + (3635.1303083215985 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4383783084459912 = (5905.72176717922 - 3316.7814487305236) / 5905.72176717922 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Feb 21, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringWithCultureInfo - Duration of single invocation 3.69 μs 2.53 μs 0.68 0.10 False
ToStringWithFormat - Duration of single invocation 4.26 μs 3.13 μs 0.74 0.07 False
ToStringWithFormat - Duration of single invocation 3.69 μs 2.57 μs 0.70 0.06 False
TryParse - Duration of single invocation 2.13 μs 1.64 μs 0.77 0.11 False
ToStringWithFormat - Duration of single invocation 3.61 μs 2.76 μs 0.76 0.08 False
ToString - Duration of single invocation 5.22 μs 3.80 μs 0.73 0.05 False
TryParse - Duration of single invocation 2.29 μs 1.82 μs 0.79 0.10 False
ToStringWithFormat - Duration of single invocation 3.82 μs 2.48 μs 0.65 0.09 False
ToString - Duration of single invocation 3.57 μs 2.56 μs 0.72 0.08 False
ToStringWithFormat - Duration of single invocation 3.58 μs 2.47 μs 0.69 0.06 False
ToStringWithFormat - Duration of single invocation 4.23 μs 3.21 μs 0.76 0.06 False
ToStringWithFormat - Duration of single invocation 3.64 μs 2.65 μs 0.73 0.10 False
Parse - Duration of single invocation 2.19 μs 1.73 μs 0.79 0.13 False
ToStringWithFormat - Duration of single invocation 5.04 μs 3.78 μs 0.75 0.05 False
Parse - Duration of single invocation 2.18 μs 1.67 μs 0.76 0.09 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.ToStringWithCultureInfo(value: 12345, culture: zh)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.5261009289707337 < 3.4054230219044035.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/13/2023 11:01:49 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 68.694466834083 (T) = (0 -2556.3784467391483) / Math.Sqrt((115023.39539561493 / (299)) + (3573.776979523308 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.37448742429211007 = (4086.8537996156283 - 2556.3784467391483) / 4086.8537996156283 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G17")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.130074422698088 < 4.081358174554273.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 58.89201205864754 (T) = (0 -3183.9642219728485) / Math.Sqrt((329714.117143836 / (299)) + (2966.4297426385647 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.39006265612487945 = (5220.1496661019955 - 3183.9642219728485) / 5220.1496661019955 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "E")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.569878267910678 < 3.555148870554391.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/13/2023 11:01:49 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 73.08581037438 (T) = (0 -2553.0388667091997) / Math.Sqrt((134553.85266858473 / (299)) + (2004.0173733242468 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.39326223714980935 = (4207.812704315834 - 2553.0388667091997) / 4207.812704315834 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.TryParse(value: "1.7976931348623157e+308")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.636459595024802 < 2.033522667109663.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 48.68380934281041 (T) = (0 -1698.4283844752667) / Math.Sqrt((116018.08925671015 / (299)) + (2322.615843961072 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.38087290917947414 = (2743.2629094364916 - 1698.4283844752667) / 2743.2629094364916 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "R")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.7554998313079526 < 3.4627358807507234.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 70.8010657345052 (T) = (0 -2617.788386020167) / Math.Sqrt((137417.74724636073 / (299)) + (2299.1564108386697 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.38404453813055034 = (4249.963752371111 - 2617.788386020167) / 4249.963752371111 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToString(value: 1.7976931348623157E+308)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.8031214291073474 < 4.951091634951551.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 48.329226143009514 (T) = (0 -3851.25611694423) / Math.Sqrt((967356.0992018753 / (299)) + (9587.415541527525 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4272949274813836 = (6724.676105988289 - 3851.25611694423) / 6724.676105988289 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.TryParse(value: "-1.7976931348623157e+308")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.8180861033519555 < 2.1050882444058057.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 45.914502421422924 (T) = (0 -1761.0906109273951) / Math.Sqrt((123393.4593458262 / (299)) + (3245.9875340081903 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3715261802861247 = (2802.170202935685 - 1761.0906109273951) / 2802.170202935685 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "E")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.4841105587175876 < 3.6906203689259574.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/13/2023 11:01:49 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 72.84656981887548 (T) = (0 -2538.052367673341) / Math.Sqrt((169863.58664779604 / (299)) + (2835.7730221964466 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4238068170580544 = (4404.863581888408 - 2538.052367673341) / 4404.863581888408 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToString(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.5561089806086557 < 3.4323875736155105.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/13/2023 11:01:49 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 67.33206703080164 (T) = (0 -2581.4380699247877) / Math.Sqrt((113831.49395892612 / (299)) + (4111.122636859162 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3705018221237911 = (4100.787199470542 - 2581.4380699247877) / 4100.787199470542 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "E")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.4680668217780375 < 3.418600333864361.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/13/2023 11:01:49 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 71.98071940756044 (T) = (0 -2464.755449419378) / Math.Sqrt((136820.36892455595 / (299)) + (2442.072111064478 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4029140419850336 = (4127.974232744554 - 2464.755449419378) / 4127.974232744554 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G17")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.205270245609934 < 4.1633688665778354.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 56.90527980621966 (T) = (0 -3229.028649302826) / Math.Sqrt((328372.59265495586 / (299)) + (6153.067999391135 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3876625055486973 = (5273.282590993815 - 3229.028649302826) / 5273.282590993815 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.6482190489477038 < 3.483753441513088.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 67.61434193183737 (T) = (0 -2611.8172494279775) / Math.Sqrt((128671.07410377682 / (299)) + (3795.881041891386 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.37754789990983495 = (4196.01323386915 - 2611.8172494279775) / 4196.01323386915 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.Parse(value: "-1.7976931348623157e+308")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7255222056300248 < 2.143002690280448.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 45.59018313967277 (T) = (0 -1765.2179013099426) / Math.Sqrt((123203.05597885093 / (299)) + (3313.1866360670238 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3696581678676967 = (2800.41369829226 - 1765.2179013099426) / 2800.41369829226 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G17")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.7806759593285735 < 4.882628923039786.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 48.83173252734865 (T) = (0 -3884.2452935326032) / Math.Sqrt((893901.2624663949 / (299)) + (11452.478334555508 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.42108641096690863 = (6709.5424379657725 - 3884.2452935326032) / 6709.5424379657725 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.Parse(value: "1.7976931348623157e+308")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6668785218467275 < 2.085568357418077.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 48.178690445527515 (T) = (0 -1723.5782440805597) / Math.Sqrt((112289.18065824719 / (299)) + (2391.6171737361788 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3723149861371962 = (2745.9286202701833 - 1723.5782440805597) / 2745.9286202701833 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 432.89 ns 225.65 ns 0.52 0.12 False
ToString - Duration of single invocation 494.63 ns 280.75 ns 0.57 0.13 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_SByte.ToString(value: 127)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 225.64953897342517 < 411.56711043733117.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 5.662383065388739 (T) = (0 -229.6036926196636) / Math.Sqrt((36204.45665991881 / (299)) + (24.678228783352676 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.213981876644255 = (292.1099218926622 - 229.6036926196636) / 292.1099218926622 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_SByte.ToString(value: -128)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 280.7481120158693 < 464.0566146893021.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 42.80224186961464 (T) = (0 -284.579504843206) / Math.Sqrt((5369.628650408463 / (299)) + (57.174506880071846 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4006004770284556 = (474.7743265333162 - 284.579504843206) / 474.7743265333162 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToUtf8Bytes - Duration of single invocation 2.99 μs 2.65 μs 0.88 0.17 False
SerializeToUtf8Bytes - Duration of single invocation 2.94 μs 2.56 μs 0.87 0.13 False
SerializeToWriter - Duration of single invocation 1.59 μs 1.26 μs 0.79 0.10 False
SerializeObjectProperty - Duration of single invocation 10.61 μs 9.56 μs 0.90 0.04 False
SerializeToWriter - Duration of single invocation 1.62 μs 1.31 μs 0.81 0.11 False

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToUtf8Bytes(Mode: Reflection)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.6472972492769027 < 2.8359931399739704.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 42.803163310546225 (T) = (0 -2602.870258134473) / Math.Sqrt((58856.32078108966 / (299)) + (3369.258743375459 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.2222980970386383 = (3346.8739734635706 - 2602.870258134473) / 3346.8739734635706 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Int32&gt;.SerializeToUtf8Bytes(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.559230332974258 < 2.874052308246769.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 41.06674274488465 (T) = (0 -2587.7157852991672) / Math.Sqrt((61167.14412449805 / (299)) + (4413.587311785446 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.22702250566473217 = (3347.724615869324 - 2587.7157852991672) / 3347.724615869324 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Int32&gt;.SerializeToWriter(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2578396364226587 < 1.5033350343757261.
IsChangePoint: Marked as a change because one of 1/15/2023 7:15:22 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 37.80562044306565 (T) = (0 -1260.770098482908) / Math.Sqrt((5580.232980376368 / (299)) + (1447.7371668783119 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.19336008431792337 = (1562.989971078767 - 1260.770098482908) / 1562.989971078767 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Int32&gt;.SerializeObjectProperty(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.557458873102926 < 10.033935082918443.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/16/2023 3:28:45 AM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 42.267503962293624 (T) = (0 -9385.489485032798) / Math.Sqrt((1203214.560963545 / (299)) + (39608.24906220084 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.24931798506713126 = (12502.616685004921 - 9385.489485032798) / 12502.616685004921 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;Int32&gt;.SerializeToWriter(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3096965538016199 < 1.539566512050024.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 34.465466904631576 (T) = (0 -1274.6633869953562) / Math.Sqrt((5626.549844254138 / (299)) + (1668.6049401093046 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.185516745354522 = (1564.9964314492652 - 1274.6633869953562) / 1564.9964314492652 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Feb 21, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FormatterInt32 - Duration of single invocation 499.00 ns 242.39 ns 0.49 0.13 False
FormatterInt64 - Duration of single invocation 682.36 ns 434.09 ns 0.64 0.06 False
FormatterUInt64 - Duration of single invocation 777.95 ns 428.71 ns 0.55 0.10 False
FormatterUInt64 - Duration of single invocation 393.64 ns 170.88 ns 0.43 0.16 False
FormatterInt32 - Duration of single invocation 414.30 ns 177.49 ns 0.43 0.18 False
FormatterInt64 - Duration of single invocation 401.78 ns 181.39 ns 0.45 0.13 False
FormatterInt32 - Duration of single invocation 502.91 ns 247.53 ns 0.49 0.11 False
FormatterInt64 - Duration of single invocation 717.15 ns 459.93 ns 0.64 0.06 False
FormatterUInt32 - Duration of single invocation 390.99 ns 172.69 ns 0.44 0.12 False
FormatterDouble - Duration of single invocation 4.50 μs 3.14 μs 0.70 0.07 False
FormatterUInt32 - Duration of single invocation 477.78 ns 242.22 ns 0.51 0.11 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 2147483647)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 242.39255477051847 < 480.17436763555264.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 41.497361521631525 (T) = (0 -259.278460168423) / Math.Sqrt((9359.540538108526 / (299)) + (352.1942088418554 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.5100550757568272 = (529.199196356479 - 259.278460168423) / 529.199196356479 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 9223372036854775807)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 434.087775203554 < 656.2196078504311.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 46.57703670961498 (T) = (0 -449.43523478119505) / Math.Sqrt((14353.064460980278 / (299)) + (468.8141317245294 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4506436715433883 = (818.1124190265728 - 449.43523478119505) / 818.1124190265728 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 18446744073709551615)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 428.7148486796343 < 660.9812448762032.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 45.94020980700543 (T) = (0 -446.6573315477818) / Math.Sqrt((16399.597604211987 / (299)) + (559.7898141965399 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.46661191530852436 = (837.396530532059 - 446.6573315477818) / 837.396530532059 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 170.87921220678948 < 377.9587236030019.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 34.95791405771989 (T) = (0 -179.28592124071156) / Math.Sqrt((10190.709417114038 / (299)) + (71.17352028093025 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.5402044453908428 = (389.9253036343752 - 179.28592124071156) / 389.9253036343752 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 177.48851174209977 < 386.87383790371854.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 36.544075488321724 (T) = (0 -187.90226118605415) / Math.Sqrt((10475.102360411713 / (299)) + (105.51306702133908 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.5462988733558142 = (414.15427503096356 - 187.90226118605415) / 414.15427503096356 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 181.3904663248954 < 385.87802106429666.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 36.581255534858556 (T) = (0 -186.88558613054872) / Math.Sqrt((10672.32786573342 / (299)) + (84.92394778702943 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.5479474968430119 = (413.415664829639 - 186.88558613054872) / 413.415664829639 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: -2147483648)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 247.5303038853111 < 475.32424932301797.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 40.59953032265723 (T) = (0 -261.64234429482696) / Math.Sqrt((9109.70669812186 / (299)) + (368.1710082575259 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.5013216029247519 = (524.6715033764465 - 261.64234429482696) / 524.6715033764465 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: -9223372036854775808)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 459.93401477793833 < 672.8465822477665.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 42.06481654856972 (T) = (0 -465.9018465037389) / Math.Sqrt((13965.237608688827 / (299)) + (1015.4408775792069 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4443310953834585 = (838.4522557101707 - 465.9018465037389) / 838.4522557101707 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 172.69422511919214 < 374.6974296553782.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 35.0505304714771 (T) = (0 -180.1292696020603) / Math.Sqrt((10254.13381254413 / (299)) + (81.41734175886576 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.5415030984691025 = (392.8691099124508 - 180.1292696020603) / 392.8691099124508 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.1430348608316567 < 4.0828146395957345.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 52.00151024450853 (T) = (0 -3202.6052641975202) / Math.Sqrt((498294.7968979277 / (299)) + (6243.741695882632 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.41237665372107607 = (5450.098748590832 - 3202.6052641975202) / 5450.098748590832 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 4294967295)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 242.2207305899662 < 454.26416921794737.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 39.08491274341814 (T) = (0 -254.2619574094553) / Math.Sqrt((9153.759933192963 / (299)) + (360.2821950145225 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4986742179014068 = (507.17909688405155 - 254.2619574094553) / 507.17909688405155 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Feb 21, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in Microsoft.Extensions.Logging.FormattingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FourArguments_DefineMessage - Duration of single invocation 5.36 μs 4.63 μs 0.86 0.07 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.FormattingOverhead*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.FormattingOverhead.FourArguments_DefineMessage

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.629121795542926 < 5.213655139793055.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 46.726305470417394 (T) = (0 -4592.505579018645) / Math.Sqrt((274871.52094199427 / (299)) + (9002.273868628969 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.26065336086438357 = (6211.572942818577 - 4592.505579018645) / 6211.572942818577 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Tests.Perf_Byte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 392.83 ns 188.42 ns 0.48 0.26 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Byte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Byte.ToString(value: 255)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 188.41572488838182 < 373.3674539330376.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 5.909086990862614 (T) = (0 -190.28770015651568) / Math.Sqrt((33970.009888177294 / (299)) + (22.0939796387846 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.24924901932728583 = (253.4631389838578 - 190.28770015651568) / 253.4631389838578 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Feb 21, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<StructRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToString - Duration of single invocation 11.41 μs 10.69 μs 0.94 0.06 False
SerializeObjectProperty - Duration of single invocation 27.48 μs 24.67 μs 0.90 0.00 False
SerializeToString - Duration of single invocation 18.67 μs 16.62 μs 0.89 0.06 False
SerializeToUtf8Bytes - Duration of single invocation 15.76 μs 13.89 μs 0.88 0.12 False
SerializeToWriter - Duration of single invocation 13.25 μs 11.52 μs 0.87 0.08 False
SerializeToUtf8Bytes - Duration of single invocation 9.86 μs 8.39 μs 0.85 0.03 False
SerializeObjectProperty - Duration of single invocation 23.07 μs 20.18 μs 0.87 0.03 False
SerializeToWriter - Duration of single invocation 7.63 μs 6.03 μs 0.79 0.03 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;StructRecord&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToString(Mode: SourceGen)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.691918296825941 < 11.376470854445113.
IsChangePoint: Marked as a change because one of 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 25.714540076995483 (T) = (0 -10427.438417218034) / Math.Sqrt((36883.054280183285 / (31)) + (68300.81784767543 / (31))) is greater than 2.0002978220134553 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (31) - 2, .975) and 0.12560388612915507 = (11925.302791039448 - 10427.438417218034) / 11925.302791039448 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;StructRecord&gt;.SerializeObjectProperty(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.668983055353024 < 25.361259169863086.
IsChangePoint: Marked as a change because one of 2/15/2023 8:55:14 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 23.276323083462014 (T) = (0 -24620.043868346675) / Math.Sqrt((167987.56325520336 / (32)) + (95920.178643233 / (31))) is greater than 1.9996235849941737 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (31) - 2, .975) and 0.07949400804341442 = (26746.20706815328 - 24620.043868346675) / 26746.20706815328 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;StructRecord&gt;.SerializeToString(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.617873316978546 < 17.457104006052788.
IsChangePoint: Marked as a change because one of 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 19.974632879335214 (T) = (0 -16402.41823247526) / Math.Sqrt((117181.42090153908 / (31)) + (122564.82967042472 / (32))) is greater than 1.9996235849941737 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (32) - 2, .975) and 0.09603318945605437 = (18144.93412939066 - 16402.41823247526) / 18144.93412939066 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;StructRecord&gt;.SerializeToUtf8Bytes(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.888629523441445 < 14.897128678491892.
IsChangePoint: Marked as a change because one of 2/15/2023 2:23:18 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 19.300532925132135 (T) = (0 -13995.217036806229) / Math.Sqrt((83493.26355238067 / (30)) + (145901.64087174612 / (33))) is greater than 1.9996235849941737 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (33) - 2, .975) and 0.1047884385339254 = (15633.418556264478 - 13995.217036806229) / 15633.418556264478 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;StructRecord&gt;.SerializeToWriter(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.518313004715015 < 12.399829174334869.
IsChangePoint: Marked as a change because one of 2/16/2023 1:00:08 AM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 17.601095205432035 (T) = (0 -11492.366741182033) / Math.Sqrt((126622.20650851658 / (34)) + (72858.02476905481 / (29))) is greater than 1.9996235849941737 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (29) - 2, .975) and 0.1078986161382178 = (12882.355020494626 - 11492.366741182033) / 12882.355020494626 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;StructRecord&gt;.SerializeToUtf8Bytes(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.387685048024894 < 9.328637200414935.
IsChangePoint: Marked as a change because one of 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 25.268843999558147 (T) = (0 -8482.558760699729) / Math.Sqrt((37645.89729013058 / (31)) + (35478.178785402466 / (32))) is greater than 1.9996235849941737 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (32) - 2, .975) and 0.12555216214517687 = (9700.47428044303 - 8482.558760699729) / 9700.47428044303 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;StructRecord&gt;.SerializeObjectProperty(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.1758037604353 < 21.615319418777965.
IsChangePoint: Marked as a change because one of 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 24.78521857223891 (T) = (0 -20555.706536253878) / Math.Sqrt((83233.55734335614 / (31)) + (128293.20643071804 / (32))) is greater than 1.9996235849941737 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (32) - 2, .975) and 0.0897938762230871 = (22583.573104252133 - 20555.706536253878) / 22583.573104252133 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;StructRecord&gt;.SerializeToWriter(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.027099984371601 < 6.919284272842227.
IsChangePoint: Marked as a change because one of 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 28.117604630037334 (T) = (0 -6246.599418705391) / Math.Sqrt((20652.847003564257 / (31)) + (28960.669381418316 / (32))) is greater than 1.9996235849941737 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (32) - 2, .975) and 0.15140985924611236 = (7361.150122668066 - 6246.599418705391) / 7361.150122668066 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Globalization.Tests.Perf_NumberCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 4.00 μs 2.88 μs 0.72 0.08 False
ToString - Duration of single invocation 4.03 μs 3.12 μs 0.77 0.07 False
ToString - Duration of single invocation 4.10 μs 2.98 μs 0.73 0.07 False
ToString - Duration of single invocation 4.48 μs 2.92 μs 0.65 0.07 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_NumberCultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: ja)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.880128353376503 < 3.8561719816107285.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 58.0807098688547 (T) = (0 -2946.5144534014894) / Math.Sqrt((279438.56670728244 / (299)) + (4251.464396764509 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3917320340763751 = (4844.105917903062 - 2946.5144534014894) / 4844.105917903062 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: da)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.1204263066417273 < 3.8139437420047404.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 60.327698105400046 (T) = (0 -2938.496565567089) / Math.Sqrt((263333.35742384405 / (299)) + (3556.0485949203344 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3926751660201938 = (4838.426491324402 - 2938.496565567089) / 4838.426491324402 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: )

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.9785034702209416 < 3.8507004099342765.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 58.58359124929351 (T) = (0 -2960.1994786349064) / Math.Sqrt((256861.9950177401 / (299)) + (5409.318427992147 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.38821588439015414 = (4838.634091838238 - 2960.1994786349064) / 4838.634091838238 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: fr)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.9233850294211194 < 3.760947094844991.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 57.617732716690625 (T) = (0 -2964.7431241640734) / Math.Sqrt((270884.7993343024 / (299)) + (4913.244075728412 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.38747574758132713 = (4840.2052856800365 - 2964.7431241640734) / 4840.2052856800365 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryFormat - Duration of single invocation 395.95 ns 169.63 ns 0.43 0.12 False
TryFormat - Duration of single invocation 579.39 ns 383.25 ns 0.66 0.13 False
TryFormat - Duration of single invocation 412.16 ns 212.98 ns 0.52 0.15 False
TryFormat - Duration of single invocation 442.44 ns 257.41 ns 0.58 0.12 False
ToString - Duration of single invocation 551.67 ns 360.13 ns 0.65 0.11 False
ToString - Duration of single invocation 512.26 ns 307.40 ns 0.60 0.10 False
ToString - Duration of single invocation 450.77 ns 255.66 ns 0.57 0.12 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.TryFormat(value: 4)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 169.62723949730562 < 353.1949365330889.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 28.09179853445334 (T) = (0 -170.73772198469737) / Math.Sqrt((8639.08789002375 / (299)) + (18.673822134200464 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4718177266562171 = (323.25530522598154 - 170.73772198469737) / 323.25530522598154 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryFormat(value: -2147483648)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 383.2533115593486 < 555.7026361654827.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 63.6937888440156 (T) = (0 -391.27045680016136) / Math.Sqrt((5249.0305142878315 / (299)) + (72.6191379274242 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.420225120546977 = (674.8661776607114 - 391.27045680016136) / 674.8661776607114 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryFormat(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 212.98009569710925 < 395.6386309966449.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 43.86427940753705 (T) = (0 -218.5916438882731) / Math.Sqrt((4294.567797478262 / (299)) + (179.3860905269798 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.472772822304864 = (414.60617573601564 - 218.5916438882731) / 414.60617573601564 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.TryFormat(value: 2147483647)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 257.4095136193995 < 422.79874387355835.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 67.3063372238686 (T) = (0 -260.65751207166977) / Math.Sqrt((3818.4826813616546 / (299)) + (32.973492158969705 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.48960831017624423 = (510.7009327712171 - 260.65751207166977) / 510.7009327712171 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToString(value: -2147483648)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 360.1336670280007 < 536.6587660518474.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/15/2023 7:15:22 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 71.05499617644118 (T) = (0 -362.46279193236353) / Math.Sqrt((4081.2527887985607 / (299)) + (73.6111299062735 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.43911919355141277 = (646.2385372525463 - 362.46279193236353) / 646.2385372525463 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToString(value: 2147483647)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 307.40101702546684 < 462.91380224656626.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 4.005936875740417 (T) = (0 -300.17741164398814) / Math.Sqrt((51802.91587675862 / (299)) + (82.01384820115378 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.15034792558429172 = (353.29450804956275 - 300.17741164398814) / 353.29450804956275 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int32.ToString(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 255.6588791622937 < 430.0128036085354.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 5.016547434342411 (T) = (0 -252.62014150002926) / Math.Sqrt((40153.45873071017 / (299)) + (40.517887059560266 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.1877887700731112 = (311.0276393528428 - 252.62014150002926) / 311.0276393528428 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringD - Duration of single invocation 892.94 ns 661.62 ns 0.74 0.15 False
Parse - Duration of single invocation 3.03 μs 2.63 μs 0.87 0.15 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.ToStringD(numberString: 123)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 661.6206862789402 < 832.5030941404796.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 19.706500381070114 (T) = (0 -652.5232529249217) / Math.Sqrt((21380.136124723354 / (299)) + (527.3126391588816 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.22074661089886877 = (837.3697978748701 - 652.5232529249217) / 837.3697978748701 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Parse(numberString: 123)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.634820307346753 < 2.708261114720192.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 41.3002006487955 (T) = (0 -2506.2164117625844) / Math.Sqrt((49182.8309081215 / (299)) + (6011.458534434475 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.23624983948844336 = (3281.4610606221427 - 2506.2164117625844) / 3281.4610606221427 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Feb 21, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 669.64 ns 415.94 ns 0.62 0.11 False
ToString - Duration of single invocation 474.60 ns 266.93 ns 0.56 0.08 False
ToString - Duration of single invocation 740.78 ns 522.68 ns 0.71 0.09 False
TryFormat - Duration of single invocation 604.84 ns 382.98 ns 0.63 0.09 False
TryFormat - Duration of single invocation 449.74 ns 236.69 ns 0.53 0.13 False
TryFormat - Duration of single invocation 769.38 ns 571.52 ns 0.74 0.07 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int64.ToString(value: 9223372036854775807)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 415.93868549182565 < 621.4179279147606.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 43.52600496344511 (T) = (0 -433.12947210967627) / Math.Sqrt((20601.935709646972 / (299)) + (954.3080532593458 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.49962264670818973 = (865.6056659244601 - 433.12947210967627) / 865.6056659244601 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ToString(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 266.93167435831555 < 452.64573226491814.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 41.972043726527446 (T) = (0 -272.5321511710247) / Math.Sqrt((4764.995589461441 / (299)) + (87.25392652246482 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.39951700622598124 = (453.8549034639062 - 272.5321511710247) / 453.8549034639062 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ToString(value: -9223372036854775808)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 522.6791127697865 < 703.0140208460778.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 49.03147084857226 (T) = (0 -531.7583352238718) / Math.Sqrt((21772.33796835567 / (299)) + (270.7267067649151 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4539328143487292 = (973.7965385883176 - 531.7583352238718) / 973.7965385883176 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryFormat(value: 9223372036854775807)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 382.97688856108897 < 573.137231467431.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 49.87460766700458 (T) = (0 -396.3870931865741) / Math.Sqrt((22586.006522452757 / (299)) + (301.5911635799137 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.5369897762329864 = (856.1087268475432 - 396.3870931865741) / 856.1087268475432 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryFormat(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 236.69110729157634 < 424.5914496068336.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 46.81686507621527 (T) = (0 -240.53176765329573) / Math.Sqrt((5127.891167362853 / (299)) + (114.13846523876119 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.469753921940801 = (453.6229075633855 - 240.53176765329573) / 453.6229075633855 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryFormat(value: -9223372036854775808)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 571.5166732666255 < 730.8339446503677.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 51.70267574623647 (T) = (0 -549.925676433482) / Math.Sqrt((20798.18451597389 / (299)) + (213.6747211359323 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4508211284421831 = (1001.3598572602523 - 549.925676433482) / 1001.3598572602523 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CtorInvoke - Duration of single invocation 336.08 μs 285.23 μs 0.85 0.02 False
Ctor - Duration of single invocation 330.19 μs 281.91 μs 0.85 0.02 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options: IgnoreCase, Compiled)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 285.2310608901515 < 319.6709159136225.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/16/2023 10:36:41 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 42.55962796213369 (T) = (0 -285889.72017364367) / Math.Sqrt((79128451.72823906 / (299)) + (20045722.378434733 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.12442772729825952 = (326517.5578156192 - 285889.72017364367) / 326517.5578156192 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 281.91303322458793 < 313.84124981971155.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/14/2023 10:23:46 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 58.48023576207747 (T) = (0 -282011.3463389505) / Math.Sqrt((75293186.4977056 / (299)) + (7549997.494819889 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.12650391334269037 = (322853.58875292743 - 282011.3463389505) / 322853.58875292743 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Tests.Perf_DateTime

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseO - Duration of single invocation 1.29 μs 855.41 ns 0.66 0.13 False
ParseR - Duration of single invocation 1.32 μs 871.38 ns 0.66 0.09 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTime*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_DateTime.ParseO

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 855.4107646753492 < 1.2095599925731928.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 63.74970933763346 (T) = (0 -882.1963484377762) / Math.Sqrt((15582.477404766585 / (299)) + (802.0951606495612 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3883211290825984 = (1442.254081973847 - 882.1963484377762) / 1442.254081973847 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_DateTime.ParseR

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 871.3772591038818 < 1.2350859392229394.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 71.98892216760815 (T) = (0 -898.2555075298708) / Math.Sqrt((6875.648499074885 / (299)) + (919.5298880452807 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.36565197605754635 = (1416.029487957163 - 898.2555075298708) / 1416.029487957163 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Tests.Perf_Int16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 498.38 ns 324.52 ns 0.65 0.12 False
ToString - Duration of single invocation 459.41 ns 250.11 ns 0.54 0.11 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int16.ToString(value: -32768)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 324.520846508329 < 487.226474619483.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 58.6517355604601 (T) = (0 -306.4648340871516) / Math.Sqrt((3600.1540926366733 / (299)) + (68.92416100255167 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.41895674121519 = (527.4389289501269 - 306.4648340871516) / 527.4389289501269 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int16.ToString(value: 32767)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 250.10634983793835 < 431.01307982283663.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 5.098391218117234 (T) = (0 -251.81158762863186) / Math.Sqrt((40176.76756482449 / (299)) + (44.1903395024454 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.19087364434966286 = (311.2141705311746 - 251.81158762863186) / 311.2141705311746 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Append_Primitives - Duration of single invocation 35.76 μs 29.01 μs 0.81 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_StringBuilder.Append_Primitives

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.005886979092335 < 33.76975187124804.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 51.05418487436259 (T) = (0 -29318.89638679534) / Math.Sqrt((19660822.055963732 / (299)) + (225560.65311894155 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3196595708104358 = (43094.44967385023 - 29318.89638679534) / 43094.44967385023 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Tests.Perf_UInt16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 408.18 ns 208.01 ns 0.51 0.23 False
ToString - Duration of single invocation 412.21 ns 209.70 ns 0.51 0.24 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt16.ToString(value: 12345)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 208.01171580342046 < 384.43554216444.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 5.408096084468054 (T) = (0 -209.4725556940583) / Math.Sqrt((38733.810757829095 / (299)) + (22.260459408806334 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.22759884271436084 = (271.19658446678625 - 209.4725556940583) / 271.19658446678625 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt16.ToString(value: 65535)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 209.69618078480568 < 382.62227799031336.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 5.324387676658646 (T) = (0 -209.7111115040278) / Math.Sqrt((38450.56350895305 / (299)) + (13.920359662611489 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.22384434510471324 = (270.1920809071738 - 209.7111115040278) / 270.1920809071738 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryFormat - Duration of single invocation 437.76 ns 242.03 ns 0.55 0.11 False
ToString - Duration of single invocation 431.74 ns 224.26 ns 0.52 0.12 False
TryFormat - Duration of single invocation 623.27 ns 393.71 ns 0.63 0.08 False
TryFormat - Duration of single invocation 401.15 ns 186.59 ns 0.47 0.15 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.TryFormat(value: 12345)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 242.02505856845647 < 426.54059769016976.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 48.63221329875258 (T) = (0 -239.5103842837423) / Math.Sqrt((4975.934650916421 / (299)) + (63.99922103019073 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4671461960380324 = (449.48611139283025 - 239.5103842837423) / 449.48611139283025 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.ToString(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 224.2575213105476 < 411.2720670471082.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 62.431974696011515 (T) = (0 -229.77062544513458) / Math.Sqrt((2765.6168249540315 / (299)) + (37.46371542264942 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.46726532411031746 = (431.3040540516926 - 229.77062544513458) / 431.3040540516926 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.TryFormat(value: 18446744073709551615)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 393.7097144301683 < 592.7430279041367.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 48.86769853653523 (T) = (0 -406.128755091912) / Math.Sqrt((25951.562270824466 / (299)) + (156.90849174112077 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.5353648956288618 = (874.0810827059402 - 406.128755091912) / 874.0810827059402 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.TryFormat(value: 0)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 186.59122641735348 < 385.56553950237617.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 27.471575857428693 (T) = (0 -193.02473801628815) / Math.Sqrt((10155.707825259453 / (299)) + (50.24443557055999 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.45899297833286545 = (356.7878609439759 - 193.02473801628815) / 356.7878609439759 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IntegerFormatting - Duration of single invocation 46.83 ms 29.29 ms 0.63 0.08 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.IntegerFormatting

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.29213388392857 < 44.18959836669643.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 4.579726032948811 (T) = (0 -28135446.770384043) / Math.Sqrt((457714301992706.56 / (299)) + (942924373999.9531 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.16896823193933572 = (33856042.39419424 - 28135446.770384043) / 33856042.39419424 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Feb 21, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 442.63 ns 254.43 ns 0.57 0.20 False
TryFormat - Duration of single invocation 406.05 ns 211.20 ns 0.52 0.11 False
ToString - Duration of single invocation 417.41 ns 216.02 ns 0.52 0.21 False
TryFormat - Duration of single invocation 361.68 ns 167.81 ns 0.46 0.10 False
TryFormat - Duration of single invocation 431.65 ns 253.94 ns 0.59 0.13 False

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.ToString(value: 4294967295)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 254.43412228631206 < 418.923678698667.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 4.4431544780196 (T) = (0 -257.28421981573814) / Math.Sqrt((49957.94968197795 / (299)) + (58.886831353537616 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.1833081319749882 = (315.03218029821085 - 257.28421981573814) / 315.03218029821085 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryFormat(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 211.1962234603076 < 390.2811604540652.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 50.35893545090582 (T) = (0 -215.1351984423032) / Math.Sqrt((4219.042174564368 / (299)) + (28.262990604468357 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.47545109936918456 = (410.133732400515 - 215.1351984423032) / 410.133732400515 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.ToString(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 216.01676557203976 < 384.32014329073667.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 5.261383882005922 (T) = (0 -210.02155862783877) / Math.Sqrt((38663.94549354941 / (299)) + (18.584536534188256 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.22210075369683305 = (269.9855535609917 - 210.02155862783877) / 269.9855535609917 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryFormat(value: 0)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 167.81439174397022 < 347.6612038580653.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 27.947249312792998 (T) = (0 -169.11723633550307) / Math.Sqrt((8466.138854850687 / (299)) + (13.015571568847228 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4696750193370676 = (318.8935888407485 - 169.11723633550307) / 318.8935888407485 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryFormat(value: 4294967295)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 253.93706523817454 < 412.7640622460722.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/15/2023 7:15:22 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 62.1199503295252 (T) = (0 -263.0088210145578) / Math.Sqrt((3829.384571053762 / (299)) + (78.87320504924 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4799840138226514 = (505.77064552946274 - 263.0088210145578) / 505.77064552946274 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in LinqBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Where01LinqMethodX - Duration of single invocation 10.06 secs 6.03 secs 0.60 0.09 False
Where01LinqQueryX - Duration of single invocation 10.08 secs 6.23 secs 0.62 0.07 False
Where01LinqMethodNestedX - Duration of single invocation 11.47 secs 7.24 secs 0.63 0.08 False
Where01ForX - Duration of single invocation 9.12 secs 5.33 secs 0.58 0.11 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'

Payloads

Baseline
Compare

Histogram

LinqBenchmarks.Where01LinqMethodX

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.027616000076923 < 9.297478968333333.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 26.266270385039 (T) = (0 -6104364845.211642) / Math.Sqrt((3.545188883087038E+18 / (299)) + (15134804584333958 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.323313144408564 = (9020959687.293352 - 6104364845.211642) / 9020959687.293352 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Where01LinqQueryX

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.229941928714286 < 9.301235512709523.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 25.50370164256883 (T) = (0 -6083446199.851127) / Math.Sqrt((3.5230580061628237E+18 / (299)) + (43694889697635896 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3246506655384034 = (9007851032.682196 - 6083446199.851127) / 9007851032.682196 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Where01LinqMethodNestedX

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.237745857214286 < 10.341082656139285.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 27.245077577845493 (T) = (0 -7203973552.683925) / Math.Sqrt((3.506205167057179E+18 / (299)) + (10607731444749486 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.29343143401721317 = (10195717584.271114 - 7203973552.683925) / 10195717584.271114 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Where01ForX

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.325438133266666 < 8.5780544241125.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 29.057732349473767 (T) = (0 -5339766914.871949) / Math.Sqrt((3.467358191159038E+18 / (299)) + (4175168329596824.5 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.370790158018907 = (8486464385.330456 - 5339766914.871949) / 8486464385.330456 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in MicroBenchmarks.Serializers.Json_FromString<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
JsonNet_ - Duration of single invocation 10.83 ms 10.12 ms 0.93 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;CollectionsOfPrimitives&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_FromString<CollectionsOfPrimitives>.JsonNet_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.121107148809523 < 10.522176749912587.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 29.348770352669913 (T) = (0 -10192622.725259967) / Math.Sqrt((570145061516.9376 / (299)) + (36049173943.53295 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.136879535609979 = (11809038.420220094 - 10192622.725259967) / 11809038.420220094 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseSingle - Duration of single invocation 1.44 μs 1.12 μs 0.78 0.14 False
TryParseDouble - Duration of single invocation 1.68 μs 1.39 μs 0.83 0.11 False
TryParseDouble - Duration of single invocation 1.71 μs 1.29 μs 0.75 0.10 False
TryParseSingle - Duration of single invocation 1.45 μs 1.15 μs 0.79 0.10 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: 3.4028235E+38)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1215166716530933 < 1.4104145692834906.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 51.47824039953551 (T) = (0 -1167.6718815956788) / Math.Sqrt((39783.84630193712 / (299)) + (1040.7066503588003 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3619557840633758 = (1830.0798791531738 - 1167.6718815956788) / 1830.0798791531738 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseDouble(value: -1.7976931348623157e+308)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3942883547022604 < 1.5575505347478757.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 43.565473295522196 (T) = (0 -1359.1948915125824) / Math.Sqrt((67640.7076491393 / (299)) + (2293.6538944766653 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3561751070113442 = (2111.1250998748374 - 1359.1948915125824) / 2111.1250998748374 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseDouble(value: 1.7976931348623157e+308)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2892137898906533 < 1.5804632795743305.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 47.85363652014691 (T) = (0 -1334.6360281682278) / Math.Sqrt((60767.83567129113 / (299)) + (1407.853330078903 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.3611813190317775 = (2089.2251086730794 - 1334.6360281682278) / 2089.2251086730794 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: -3.4028235E+38)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1491494620678582 < 1.4330266457326675.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 48.00708514590221 (T) = (0 -1179.489455156587) / Math.Sqrt((40265.06799900357 / (299)) + (1939.5840863508145 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3625584653154858 = (1850.349233581627 - 1179.489455156587) / 1850.349233581627 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Globalization.Tests.Perf_DateTimeCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 11.07 μs 9.05 μs 0.82 0.06 False
Parse - Duration of single invocation 11.23 μs 9.38 μs 0.84 0.03 False
ToStringHebrewIsrael - Duration of single invocation 8.45 μs 6.96 μs 0.82 0.04 False
Parse - Duration of single invocation 10.46 μs 8.97 μs 0.86 0.04 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_DateTimeCultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: fr)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.054817393775757 < 10.146088797776992.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 44.44631788464351 (T) = (0 -9184.25123207575) / Math.Sqrt((1002782.4699226674 / (299)) + (53033.457111237025 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.25516360222505324 = (12330.561797881934 - 9184.25123207575) / 12330.561797881934 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: ja)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.376645492788462 < 10.208023667061555.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 45.34609139002679 (T) = (0 -9357.979807784659) / Math.Sqrt((937834.7372575678 / (299)) + (39906.96647570455 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.24290089035124102 = (12360.31014767156 - 9357.979807784659) / 12360.31014767156 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToStringHebrewIsrael

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.955326885319055 < 8.292202927761384.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 61.69269135822265 (T) = (0 -6997.563688716707) / Math.Sqrt((372660.75962998567 / (299)) + (11909.260495914092 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.26182246631338313 = (9479.513219224342 - 6997.563688716707) / 9479.513219224342 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: )

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.968354142790925 < 10.0651368490283.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 44.14009134959792 (T) = (0 -9287.650889418446) / Math.Sqrt((1002943.5525890511 / (299)) + (39767.37332976999 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.24450352052870392 = (12293.440329355388 - 9287.650889418446) / 12293.440329355388 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Feb 21, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Xml.Tests.Perf_XmlConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DateTime_ToString_Unspecified - Duration of single invocation 1.56 μs 1.35 μs 0.87 0.09 False
TimeSpan_ToString - Duration of single invocation 3.35 μs 2.39 μs 0.72 0.10 False
DateTime_ToString - Duration of single invocation 1.65 μs 1.47 μs 0.89 0.12 False
DateTime_ToString_RoundtripKind - Duration of single invocation 1.60 μs 1.26 μs 0.79 0.12 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Tests.Perf_XmlConvert*'

Payloads

Baseline
Compare

Histogram

System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_Unspecified

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3534812397565912 < 1.4881574541468774.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 44.60460290178658 (T) = (0 -1352.0016271606269) / Math.Sqrt((21151.52647068519 / (299)) + (1107.2098430381232 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.2529572131685755 = (1809.8048076939876 - 1352.0016271606269) / 1809.8048076939876 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Xml.Tests.Perf_XmlConvert.TimeSpan_ToString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.3940683912091347 < 3.072638394569297.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 41.48376775085569 (T) = (0 -2353.1247593733387) / Math.Sqrt((106141.24992421124 / (299)) + (3198.138320629756 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.2732637433502055 = (3237.9349975204022 - 2353.1247593733387) / 3237.9349975204022 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.4666212571001862 < 1.6114730194558464.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 38.87236723120847 (T) = (0 -1449.1787327632624) / Math.Sqrt((24140.413670249894 / (299)) + (2200.24780272189 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.24697013764618608 = (1924.4638296726143 - 1449.1787327632624) / 1924.4638296726143 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_RoundtripKind

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2560475239046667 < 1.4615772044044713.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 47.323880019314124 (T) = (0 -1303.9814428453487) / Math.Sqrt((18487.832438654874 / (299)) + (1017.8007219934722 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.25990897108616995 = (1761.920347499812 - 1303.9814428453487) / 1761.920347499812 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetDateTimeOffset - Duration of single invocation 163.86 μs 126.20 μs 0.77 0.08 False
GetDateTime - Duration of single invocation 110.16 μs 79.84 μs 0.72 0.10 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Get.GetDateTimeOffset

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 126.19516179435483 < 155.50964940627296.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 42.650914906608136 (T) = (0 -133344.81309902828) / Math.Sqrt((552802205.8975188 / (299)) + (22819680.3686951 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.33860770437825105 = (201612.28665912422 - 133344.81309902828) / 201612.28665912422 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Get.GetDateTime

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 79.83771264880953 < 104.65254473048394.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 42.97696773149654 (T) = (0 -80932.95361956627) / Math.Sqrt((334923074.620709 / (299)) + (19375622.45965729 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4109208388419487 = (137388.9265756114 - 80932.95361956627) / 137388.9265756114 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Basic

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteBasicUtf8 - Duration of single invocation 69.62 ms 44.11 ms 0.63 0.08 False
WriteBasicUtf8 - Duration of single invocation 57.05 ms 34.69 ms 0.61 0.06 False
WriteBasicUtf8 - Duration of single invocation 53.21 ms 32.15 ms 0.60 0.07 False
WriteBasicUtf8 - Duration of single invocation 64.47 ms 40.88 ms 0.63 0.09 False
WriteBasicUtf16 - Duration of single invocation 55.70 ms 34.18 ms 0.61 0.07 False
WriteBasicUtf16 - Duration of single invocation 63.82 ms 41.12 ms 0.64 0.09 False
WriteBasicUtf16 - Duration of single invocation 52.65 ms 31.07 ms 0.59 0.07 False
WriteBasicUtf16 - Duration of single invocation 67.72 ms 44.30 ms 0.65 0.07 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Basic*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: False, DataSize: 100000)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 44.106016733333334 < 65.91775910410715.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 47.42588972018171 (T) = (0 -45669436.6122836) / Math.Sqrt((97551185278636.03 / (299)) + (2926640748100.3203 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.40246152434800925 = (76429281.91770817 - 45669436.6122836) / 76429281.91770817 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: False, DataSize: 100000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.69470718571428 < 53.089475408750005.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 42.44745139077078 (T) = (0 -34986424.702587) / Math.Sqrt((95583147207651.83 / (299)) + (1150928818692.5767 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4197967675503101 = (60300292.631721444 - 34986424.702587) / 60300292.631721444 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: True, DataSize: 100000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.14553335238095 < 50.74517171148809.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 41.785842140138506 (T) = (0 -31825071.59730793) / Math.Sqrt((96403538037631.83 / (299)) + (1875857104300.8801 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4476619835683657 = (57618832.400697306 - 31825071.59730793) / 57618832.400697306 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: True, DataSize: 100000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 40.882566677777774 < 61.42019169833333.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 44.77130851402237 (T) = (0 -42074159.08350372) / Math.Sqrt((101278622772977.38 / (299)) + (4576698224301.646 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.42481570334254654 = (73149005.15888156 - 42074159.08350372) / 73149005.15888156 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: False, DataSize: 100000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.1776043956044 < 53.72204084125.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 43.70650991355786 (T) = (0 -35115825.440978214) / Math.Sqrt((93169950246268.25 / (299)) + (738565695317.6104 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.418620964187583 = (60400914.51166189 - 35115825.440978214) / 60400914.51166189 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: True, DataSize: 100000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 41.122166666666665 < 60.64740628166666.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 45.283797465957264 (T) = (0 -41867433.72712435) / Math.Sqrt((106449210288286.14 / (299)) + (4062235595162.9443 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4291138967678175 = (73337629.85310686 - 41867433.72712435) / 73337629.85310686 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: True, DataSize: 100000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.06740178571429 < 52.267812507916666.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 43.23319140638064 (T) = (0 -31641962.428384595) / Math.Sqrt((96714209152103.64 / (299)) + (1226334317750.2747 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4510986721323021 = (57645993.5910581 - 31641962.428384595) / 57645993.5910581 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: False, DataSize: 100000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 44.3041999 < 65.58655920625.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 47.4512983237601 (T) = (0 -45620860.607650466) / Math.Sqrt((99151865116487.17 / (299)) + (2819576590868.4644 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.40338841155849753 = (76466601.54024744 - 45620860.607650466) / 76466601.54024744 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Deep

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteDeepUtf16 - Duration of single invocation 209.58 ms 178.11 ms 0.85 0.09 False
WriteDeepUtf8 - Duration of single invocation 161.69 ms 136.94 ms 0.85 0.08 False
WriteDeepUtf16 - Duration of single invocation 161.83 ms 131.99 ms 0.82 0.08 False
WriteDeepUtf8 - Duration of single invocation 205.32 ms 171.62 ms 0.84 0.06 False
WriteDeepUtf16 - Duration of single invocation 210.04 ms 182.86 ms 0.87 0.06 False
WriteDeepUtf8 - Duration of single invocation 217.83 ms 184.57 ms 0.85 0.04 False
WriteDeepUtf8 - Duration of single invocation 151.73 ms 115.36 ms 0.76 0.10 False
WriteDeepUtf16 - Duration of single invocation 153.98 ms 116.50 ms 0.76 0.08 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Deep*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: True, SkipValidation: True)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 178.1129233076923 < 197.44244615384616.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 46.24787501429176 (T) = (0 -174033033.61601424) / Math.Sqrt((590141282437408.4 / (299)) + (16126320741334.824 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.29492741407374234 = (246829953.52511984 - 174033033.61601424) / 246829953.52511984 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 136.93970829166665 < 153.23155321121794.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 51.82318186071329 (T) = (0 -131273785.4093836) / Math.Sqrt((270243709810251.38 / (299)) + (7103640862135.358 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.29521385002597883 = (186260449.94530955 - 131273785.4093836) / 186260449.94530955 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 131.99329166666666 < 150.60731967291665.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 52.184560760888466 (T) = (0 -131498952.43159053) / Math.Sqrt((261655074607969.72 / (299)) + (6353271396549.78 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.2913813366141733 = (185570828.46686518 - 131498952.43159053) / 185570828.46686518 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: True, SkipValidation: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 171.61746146153845 < 195.1429924375.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 48.45895143583466 (T) = (0 -172483802.39921013) / Math.Sqrt((622444108004310.5 / (299)) + (9284588438860.549 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3020106878853663 = (247115248.62272158 - 172483802.39921013) / 247115248.62272158 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: True, SkipValidation: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 182.860923 < 201.72350530560897.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 47.466970580975115 (T) = (0 -183641769.0101341) / Math.Sqrt((554843385397492.5 / (299)) + (11771799790052.938 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.27873542774490206 = (254610826.69839412 - 183641769.0101341) / 254610826.69839412 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: True, SkipValidation: False)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 184.5676152307692 < 207.83709331987177.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 47.26629527432071 (T) = (0 -183313959.90596378) / Math.Sqrt((555421771794829.9 / (299)) + (12929401858821.402 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.2794165889659856 = (254396586.29236835 - 183313959.90596378) / 254396586.29236835 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 115.3601785 < 141.80429725535714.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 51.52559975257082 (T) = (0 -117204512.40229796) / Math.Sqrt((319459940273887.3 / (299)) + (12536081576509.23 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3469299295020148 = (179466978.65502557 - 117204512.40229796) / 179466978.65502557 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: True)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 116.49615384615385 < 144.94783949615382.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 48.99089175153843 (T) = (0 -118662053.34041896) / Math.Sqrt((307391948421366.6 / (299)) + (16133703772778.453 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3382106918231148 = (179304881.28814343 - 118662053.34041896) / 179304881.28814343 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@lewing
Copy link
Member

lewing commented Feb 21, 2023 •
edited
Loading

This is mostly from dotnet/runtime#82093 cc @BrzVlad @SamMonoRT

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-wasm arch-x64 branch-refs/heads/main kind-micro os-linux perf-improvement untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@lewing