[Perf] Changes at 5/30/2022 5:18:40 PM #70019

performanceautofiler · 2022-05-31T12:22:46Z

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Tests.Perf_UInt32

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Parse - Duration of single invocation	18.02 ns	20.91 ns	1.16	0.04	False
ParseSpan - Duration of single invocation	12.60 ns	13.90 ns	1.10	0.04	False
TryParse - Duration of single invocation	18.60 ns	20.90 ns	1.12	0.02	False
TryFormat - Duration of single invocation	10.07 ns	14.28 ns	1.42	0.02	False
Parse - Duration of single invocation	12.48 ns	14.67 ns	1.18	0.05	False
TryFormat - Duration of single invocation	6.22 ns	8.87 ns	1.43	0.08	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.Parse(value: "4294967295")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.90963580322242 > 18.784444736706895.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -12.874258442904152 (T) = (0 -20.685657621666795) / Math.Sqrt((0.3930459224307238 / (63)) + (0.12439460999108248 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.15731933657549754 = (17.873768257320886 - 20.685657621666795) / 17.873768257320886 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.ParseSpan(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.904170501323676 > 13.446475611467925.
IsChangePoint: Marked as a change because one of 4/8/2022 10:19:59 AM, 4/16/2022 5:36:25 PM, 5/30/2022 9:21:47 AM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -23.580213507568608 (T) = (0 -13.949503114739542) / Math.Sqrt((0.0714804080330334 / (61)) + (0.011735460891892903 / (5))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (61) + (5) - 2, .025) and -0.11145069804060415 = (12.550716949776868 - 13.949503114739542) / 12.550716949776868 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryParse(value: "4294967295")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.90375599341185 > 19.736056437729772.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -12.10778235242679 (T) = (0 -20.64415202822326) / Math.Sqrt((0.23215992784562625 / (63)) + (0.09534218330846458 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.12416607372545044 = (18.36396997803821 - 20.64415202822326) / 18.36396997803821 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryFormat(value: 4294967295)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.282920456766071 > 10.581393941692605.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/5/2022 12:13:02 AM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -160.85537211932368 (T) = (0 -14.246516574728334) / Math.Sqrt((0.0259529440371242 / (63)) + (0.0009960487206765827 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.44501742446885745 = (9.859062135506706 - 14.246516574728334) / 9.859062135506706 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.Parse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.667848528809003 > 13.221865998475938.
IsChangePoint: Marked as a change because one of 3/26/2022 4:52:22 AM, 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -13.61347973216338 (T) = (0 -14.532805035709623) / Math.Sqrt((0.0620460584752799 / (63)) + (0.05094260911968003 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.14358556218507731 = (12.70810468080887 - 14.532805035709623) / 12.70810468080887 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryFormat(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.874208546011742 > 6.533805810690146.
IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -37.744626161663 (T) = (0 -9.017894593080138) / Math.Sqrt((0.061713030120306656 / (63)) + (0.01775431853011536 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.5328427078851965 = (5.883118043808798 - 9.017894593080138) / 5.883118043808798 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Tests.Perf_Int64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryParseSpan - Duration of single invocation	29.28 ns	35.90 ns	1.23	0.02	False
TryParse - Duration of single invocation	28.96 ns	36.10 ns	1.25	0.02	False
TryParse - Duration of single invocation	28.67 ns	35.58 ns	1.24	0.00	False
TryParse - Duration of single invocation	13.85 ns	15.26 ns	1.10	0.02	False
ParseSpan - Duration of single invocation	29.42 ns	36.85 ns	1.25	0.01	False
ParseSpan - Duration of single invocation	29.17 ns	36.29 ns	1.24	0.01	False
Parse - Duration of single invocation	29.15 ns	35.93 ns	1.23	0.01	False
Parse - Duration of single invocation	28.73 ns	35.85 ns	1.25	0.00	False
Parse - Duration of single invocation	13.71 ns	14.99 ns	1.09	0.03	False
ParseSpan - Duration of single invocation	14.11 ns	15.13 ns	1.07	0.01	False
TryParseSpan - Duration of single invocation	29.44 ns	36.22 ns	1.23	0.00	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int64.TryParseSpan(value: "9223372036854775807")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.900716326473166 > 30.429464294572007.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -73.05297825945446 (T) = (0 -35.907973090939) / Math.Sqrt((0.5706683460185307 / (62)) + (4.6626617063356646E-05 / (3))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (62) + (3) - 2, .025) and -0.24277360223567693 = (28.893414718773123 - 35.907973090939) / 28.893414718773123 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryParse(value: "-9223372036854775808")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.09783981842236 > 30.554607480463783.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -66.8531781225283 (T) = (0 -35.9529600503081) / Math.Sqrt((0.14025398858402163 / (63)) + (0.023082790265517565 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.22730376522629492 = (29.29426362810755 - 35.9529600503081) / 29.29426362810755 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryParse(value: "9223372036854775807")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.58142235169034 > 30.234881416460922.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -83.2199470412605 (T) = (0 -35.6899605880321) / Math.Sqrt((0.07790683962682238 / (63)) + (0.015909159782171292 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.23238340922205664 = (28.960111212923117 - 35.6899605880321) / 28.960111212923117 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryParse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.261934127988493 > 14.483981674952444.
IsChangePoint: Marked as a change because one of 3/26/2022 4:52:22 AM, 3/30/2022 6:37:35 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -29.580994960848166 (T) = (0 -15.289749918526136) / Math.Sqrt((0.05031466535805899 / (63)) + (0.00537326502029719 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.1092075814195988 = (13.784390022792516 - 15.289749918526136) / 13.784390022792516 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ParseSpan(value: "-9223372036854775808")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.84727049929392 > 30.9086275873624.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -61.244756338756474 (T) = (0 -36.696407848342055) / Math.Sqrt((0.05749269576901382 / (62)) + (0.040243882518615326 / (3))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (62) + (3) - 2, .025) and -0.24979800756393505 = (29.36187097935088 - 36.696407848342055) / 29.36187097935088 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ParseSpan(value: "9223372036854775807")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.29325311558419 > 30.5373217957792.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -190.03316591378754 (T) = (0 -36.28182081910127) / Math.Sqrt((0.05515825758478877 / (62)) + (0.0017737955038416974 / (3))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (62) + (3) - 2, .025) and -0.2524429686785497 = (28.96884067893499 - 36.28182081910127) / 28.96884067893499 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.Parse(value: "-9223372036854775808")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.92793820701734 > 30.631258337420388.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -22.17634450076978 (T) = (0 -36.13059356432015) / Math.Sqrt((0.10494217907618374 / (63)) + (0.2656123200248028 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.22600500179268837 = (29.470184470282987 - 36.13059356432015) / 29.470184470282987 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.Parse(value: "9223372036854775807")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.854973534418235 > 30.160064026406275.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -48.9795648018665 (T) = (0 -35.832072835745805) / Math.Sqrt((0.1487416779463161 / (63)) + (0.04823235737302959 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.22791523998032076 = (29.181226577430596 - 35.832072835745805) / 29.181226577430596 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.Parse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.988327610326593 > 14.418160884654744.
IsChangePoint: Marked as a change because one of 3/26/2022 4:52:22 AM, 3/30/2022 6:37:35 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -25.158217370356574 (T) = (0 -15.078662593838393) / Math.Sqrt((0.03530098336008355 / (63)) + (0.0065332247519817 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.09565640768781127 = (13.762218235604754 - 15.078662593838393) / 13.762218235604754 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ParseSpan(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.133668691470513 > 14.799796602882415.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -7.896044870754496 (T) = (0 -15.421821462857048) / Math.Sqrt((0.04045834132847233 / (62)) + (0.12558167550094462 / (3))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (62) + (3) - 2, .025) and -0.11802891738947521 = (13.793759019101222 - 15.421821462857048) / 13.793759019101222 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryParseSpan(value: "-9223372036854775808")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.21639956941242 > 30.92121567458197.
IsChangePoint: Marked as a change because one of 4/21/2022 3:06:05 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -51.92687470020792 (T) = (0 -36.35896690761533) / Math.Sqrt((0.09227762044301657 / (62)) + (0.053350921379235965 / (3))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (62) + (3) - 2, .025) and -0.24729341424862317 = (29.150291737504432 - 36.35896690761533) / 29.150291737504432 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.CreateAddAndClear<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Dictionary - Duration of single invocation	14.38 μs	17.12 μs	1.19	0.01	False
Span - Duration of single invocation	1.09 μs	1.27 μs	1.17	0.41	False
IDictionary - Duration of single invocation	15.69 μs	18.64 μs	1.19	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<String>.Dictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.117625870364577 > 14.93440068308716.
IsChangePoint: Marked as a change because one of 3/20/2022 12:44:50 AM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -20.604820487853146 (T) = (0 -16981.715544424256) / Math.Sqrt((21146.982497953864 / (63)) + (51471.054983084076 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.19115401617742828 = (14256.52376921067 - 16981.715544424256) / 14256.52376921067 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;String&gt;.Span(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2705983781760568 > 1.171387688564955.
IsChangePoint: Marked as a change because one of 5/1/2022 10:22:11 AM, 5/7/2022 5:14:33 AM, 5/27/2022 6:33:44 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -9.40347035776082 (T) = (0 -1348.54392768287) / Math.Sqrt((5185.886876618465 / (48)) + (3293.5958691364126 / (18))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (18) - 2, .025) and -0.1350156056116734 = (1188.1280935834566 - 1348.54392768287) / 1188.1280935834566 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;String&gt;.IDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.640563153698977 > 16.3060764320747.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -148.06350576110805 (T) = (0 -18617.460411816468) / Math.Sqrt((18390.8119899223 / (63)) + (422.0202055953375 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.19819563663150466 = (15537.913711784044 - 18617.460411816468) / 15537.913711784044 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Text.Json.Tests.Perf_Booleans

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
WriteBooleans - Duration of single invocation	1.28 ms	1.41 ms	1.10	0.03	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Booleans*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Booleans.WriteBooleans(Formatted: False, SkipValidation: False)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4083138362847225 > 1.3371767802884615.
IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/20/2022 10:18:01 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -19.000157394402965 (T) = (0 -1409275.8037444036) / Math.Sqrt((2465533646.103263 / (62)) + (701569.5314954243 / (3))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (62) + (3) - 2, .025) and -0.09321824855532522 = (1289107.4637719821 - 1409275.8037444036) / 1289107.4637719821 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.AddGivenSize<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IDictionary - Duration of single invocation	11.43 μs	13.80 μs	1.21	0.01	False
Dictionary - Duration of single invocation	10.31 μs	12.54 μs	1.22	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<String>.IDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.798332819383258 > 12.003789028223215.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -73.01182758091328 (T) = (0 -13750.654563387116) / Math.Sqrt((19799.46109350553 / (63)) + (1767.874673691752 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.18991870092908086 = (11555.961388497124 - 13750.654563387116) / 11555.961388497124 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.AddGivenSize&lt;String&gt;.Dictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.54316092908695 > 10.837595829547027.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -47.4391453766967 (T) = (0 -12511.993589076936) / Math.Sqrt((13933.601442032314 / (63)) + (5380.782194242741 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.20508843688281608 = (10382.635171109532 - 12511.993589076936) / 10382.635171109532 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler · 2022-05-31T12:22:50Z

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Tests.Perf_Int16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	11.94 ns	12.98 ns	1.09	0.13	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int16.ToString(value: 32767)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.982022617391747 > 12.477142314937668.
IsChangePoint: Marked as a change because one of 3/31/2022 9:50:44 PM, 4/11/2022 1:47:52 PM, 5/5/2022 12:13:02 AM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -12.538564411845678 (T) = (0 -12.84031180503044) / Math.Sqrt((0.07645703121413368 / (63)) + (0.03789916454959699 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.12982418447018823 = (11.364876041356545 - 12.84031180503044) / 11.364876041356545 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.TryAddGiventSize<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	10.37 μs	12.48 μs	1.20	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryAddGiventSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryAddGiventSize<String>.Dictionary(Count: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.477043549456377 > 10.81402088050692.
IsChangePoint: Marked as a change because one of 4/15/2022 3:08:53 AM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -43.959816361284624 (T) = (0 -12450.61276000739) / Math.Sqrt((27266.55147801572 / (62)) + (5083.406261127095 / (3))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (62) + (3) - 2, .025) and -0.19490412809614754 = (10419.758763278418 - 12450.61276000739) / 10419.758763278418 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Text.Json.Document.Tests.Perf_DocumentParse

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse - Duration of single invocation	189.53 ns	206.86 ns	1.09	0.20	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: HelloWorld)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 206.85502751993832 > 194.34641765214369.
IsChangePoint: Marked as a change because one of 5/13/2022 2:49:07 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -4.8274589842206375 (T) = (0 -211.64463511660486) / Math.Sqrt((36.81654958104719 / (62)) + (66.35771154202067 / (3))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (62) + (3) - 2, .025) and -0.12196284790064184 = (188.63782834932834 - 211.64463511660486) / 188.63782834932834 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2022-05-31T12:22:50Z

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Tests.Perf_UInt64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ParseSpan - Duration of single invocation	23.26 ns	33.75 ns	1.45	0.35	False
Parse - Duration of single invocation	24.60 ns	33.60 ns	1.37	0.39	False
TryParse - Duration of single invocation	22.55 ns	33.78 ns	1.50	0.31	False
TryParse - Duration of single invocation	12.21 ns	14.73 ns	1.21	0.05	False
Parse - Duration of single invocation	13.49 ns	14.87 ns	1.10	0.05	False
ParseSpan - Duration of single invocation	12.81 ns	15.17 ns	1.18	0.06	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.ParseSpan(value: "18446744073709551615")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.746031580696474 > 24.44350992377661.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -70.0490237703976 (T) = (0 -33.73528295514424) / Math.Sqrt((1.3473812100023632 / (63)) + (0.0011885591195303369 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.4418847524364928 = (23.396656978401676 - 33.73528295514424) / 23.396656978401676 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.Parse(value: "18446744073709551615")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.60414845875554 > 24.983724315819735.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -82.07512179228777 (T) = (0 -33.765518351827154) / Math.Sqrt((0.6431324760499566 / (63)) + (0.019708088779062487 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.4595376775864337 = (23.134393082379034 - 33.765518351827154) / 23.134393082379034 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.TryParse(value: "18446744073709551615")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.77764738113385 > 23.26489405094771.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -55.06880654191043 (T) = (0 -33.763953042160544) / Math.Sqrt((1.696139461447148 / (62)) + (0.03267394105417805 / (3))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (62) + (3) - 2, .025) and -0.468378640560008 = (22.994037171014487 - 33.763953042160544) / 22.994037171014487 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.TryParse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.728835842833268 > 12.83270634404727.
IsChangePoint: Marked as a change because one of 4/10/2022 7:04:55 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -17.13187114825677 (T) = (0 -14.567987361164995) / Math.Sqrt((0.09139928877995147 / (63)) + (0.058618537124966584 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.20536879554023388 = (12.085917119362438 - 14.567987361164995) / 12.085917119362438 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.Parse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.866483715251206 > 13.671465027667114.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -12.204793808098305 (T) = (0 -14.747524520646309) / Math.Sqrt((0.1746146867827679 / (63)) + (0.06987899274641593 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.15421395144979294 = (12.777115111216718 - 14.747524520646309) / 12.777115111216718 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt64.ParseSpan(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.167536629118015 > 13.320632576134148.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -15.018062366649056 (T) = (0 -14.954942532684905) / Math.Sqrt((0.04503593677644969 / (63)) + (0.06494217085987729 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.17670769551904117 = (12.709139737620513 - 14.954942532684905) / 12.709139737620513 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
EncodeUtf8 - Duration of single invocation	11.48 ns	13.18 ns	1.15	0.03	False
EncodeUtf8 - Duration of single invocation	11.41 ns	13.54 ns	1.19	0.04	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,no escaping required,16)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.181106702541285 > 12.048821643664594.
IsChangePoint: Marked as a change because one of 5/11/2022 11:37:12 AM, 5/25/2022 1:42:32 AM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -24.085096798676023 (T) = (0 -13.214685655288974) / Math.Sqrt((0.12521836667498476 / (63)) + (0.0009193536292192249 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.0956450125652133 = (12.061101455068624 - 13.214685655288974) / 12.061101455068624 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,no &lt;escaping /&gt; required,16)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.541082146028877 > 11.942176556119243.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/13/2022 5:30:39 PM, 5/11/2022 11:37:12 AM, 5/25/2022 1:42:32 AM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -23.699899998993487 (T) = (0 -13.546252674882112) / Math.Sqrt((0.19350975737925807 / (62)) + (0.001099557430615689 / (3))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (62) + (3) - 2, .025) and -0.11522772556820328 = (12.146624733509347 - 13.546252674882112) / 12.146624733509347 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Tests.Perf_UInt16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Parse - Duration of single invocation	12.52 ns	14.45 ns	1.15	0.06	False
Parse - Duration of single invocation	12.38 ns	14.47 ns	1.17	0.03	False
TryParse - Duration of single invocation	12.77 ns	14.28 ns	1.12	0.06	False
TryParse - Duration of single invocation	12.77 ns	14.25 ns	1.12	0.04	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt16.Parse(value: "12345")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.449937232534413 > 13.197572692386982.
IsChangePoint: Marked as a change because one of 3/31/2022 1:45:25 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -22.397116430697334 (T) = (0 -14.478386492763088) / Math.Sqrt((0.316975109257349 / (62)) + (0.0006693513579912628 / (3))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (62) + (3) - 2, .025) and -0.1273910526074617 = (12.842381939502774 - 14.478386492763088) / 12.842381939502774 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt16.Parse(value: "65535")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.47404754985135 > 13.270522904189766.
IsChangePoint: Marked as a change because one of 3/31/2022 1:45:25 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -52.16457527782907 (T) = (0 -14.488075099141433) / Math.Sqrt((0.04976414011371162 / (63)) + (0.000833728868508577 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.13334455468650333 = (12.783469104105775 - 14.488075099141433) / 12.783469104105775 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt16.TryParse(value: "65535")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.27971237400856 > 13.358781764564576.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 10:25:35 AM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -18.647737202554435 (T) = (0 -14.240886713345272) / Math.Sqrt((0.053638666111133 / (62)) + (0.02023621520532068 / (4))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (62) + (4) - 2, .025) and -0.11208333987846454 = (12.805593072639327 - 14.240886713345272) / 12.805593072639327 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt16.TryParse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.247554377872051 > 13.379991432947596.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -6.1187251414374755 (T) = (0 -13.948854693820046) / Math.Sqrt((0.47298927566257715 / (63)) + (0.06706601974038834 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.08202110496032403 = (12.891481164160405 - 13.948854693820046) / 12.891481164160405 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.IterateForEach<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
HashSet - Duration of single invocation	1.04 μs	1.16 μs	1.12	0.04	False
Dictionary - Duration of single invocation	1.10 μs	1.28 μs	1.17	0.32	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<Int32>.HashSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1622879129258008 > 1.0886485942587834.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -30.143027714217432 (T) = (0 -1162.2601007405333) / Math.Sqrt((965.1942283689378 / (63)) + (1.1254130894513712 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.11451416715887551 = (1042.8401315914825 - 1162.2601007405333) / 1042.8401315914825 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.Dictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2831615944734875 > 1.1493564297906371.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -13.43377553790511 (T) = (0 -1270.0807567040445) / Math.Sqrt((478.916170262833 / (63)) + (549.9407088725135 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.1711605543577881 = (1084.463399982336 - 1270.0807567040445) / 1084.463399982336 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.CtorFromCollection<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedDictionaryDeepCopy - Duration of single invocation	11.74 μs	12.65 μs	1.08	0.06	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorFromCollection&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<String>.SortedDictionaryDeepCopy(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.653068722293332 > 12.374477364805838.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -8.72345032686374 (T) = (0 -12720.19259337078) / Math.Sqrt((110163.35370526396 / (63)) + (25489.72915806504 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.07459314420442087 = (11837.217333810764 - 12720.19259337078) / 11837.217333810764 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Tests.Perf_Double

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryParse - Duration of single invocation	96.58 ns	111.51 ns	1.15	0.01	False
Parse - Duration of single invocation	96.53 ns	118.48 ns	1.23	0.01	False
Parse - Duration of single invocation	93.49 ns	115.20 ns	1.23	0.03	False
TryParse - Duration of single invocation	94.24 ns	107.22 ns	1.14	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.TryParse(value: "-1.7976931348623157e+308")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 111.50569476155883 > 101.6118099309403.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -14.808727802043872 (T) = (0 -110.55528552583282) / Math.Sqrt((0.3512375747775569 / (63)) + (2.5790333338269122 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.14233184351147318 = (96.78035866179759 - 110.55528552583282) / 96.78035866179759 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.Parse(value: "-1.7976931348623157e+308")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 118.47921777561497 > 101.1887952204233.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -276.9582851638415 (T) = (0 -118.55587231000918) / Math.Sqrt((0.1582796406113081 / (62)) + (0.011680950220784981 / (3))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (62) + (3) - 2, .025) and -0.230869717735914 = (96.31878224129454 - 118.55587231000918) / 96.31878224129454 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.Parse(value: "1.7976931348623157e+308")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 115.1953667892145 > 98.17697140397148.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -156.3895160376108 (T) = (0 -115.2112559529723) / Math.Sqrt((1.0290146936888598 / (63)) + (0.006642132726648491 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.22679194629380886 = (93.91262821787382 - 115.2112559529723) / 93.91262821787382 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.TryParse(value: "1.7976931348623157e+308")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 107.21676421950731 > 98.60884375861498.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -19.977452328253275 (T) = (0 -108.63799389484491) / Math.Sqrt((1.0963194960307667 / (63)) + (1.5191199002781552 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.1535163290247701 = (94.17984918054165 - 108.63799389484491) / 94.17984918054165 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.ContainsKeyFalse<String, String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableDictionary - Duration of single invocation	24.78 μs	26.50 μs	1.07	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyFalse&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<String, String>.ImmutableDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.503872617036684 > 25.631221750916286.
IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -20.19492912686905 (T) = (0 -26560.33635085671) / Math.Sqrt((162675.97559564954 / (63)) + (12028.90785001433 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.06579360706897917 = (24920.712767174347 - 26560.33635085671) / 24920.712767174347 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.TryGetValueFalse<String, String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableDictionary - Duration of single invocation	26.26 μs	29.10 μs	1.11	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueFalse&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<String, String>.ImmutableDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.096443629075008 > 28.118424327141213.
IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -17.096946013847795 (T) = (0 -29232.008189087155) / Math.Sqrt((153727.11550349425 / (63)) + (76508.7472019438 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.10836234361666154 = (26374.053898025017 - 29232.008189087155) / 26374.053898025017 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.TryAddDefaultSize<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	13.57 μs	16.82 μs	1.24	0.00	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryAddDefaultSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryAddDefaultSize<String>.Dictionary(Count: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.82061724157559 > 14.398362722215348.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -21.82868875748869 (T) = (0 -16581.371976313694) / Math.Sqrt((25078.341212240357 / (63)) + (44851.15368063116 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.19487813325065637 = (13877.040272888922 - 16581.371976313694) / 13877.040272888922 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

dotnet-issue-labeler · 2022-05-31T16:21:01Z

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

tannergooding · 2022-05-31T16:22:39Z

Possibly related to #69878

ghost · 2022-06-01T15:40:09Z

Tagging subscribers to this area: @JulieLeeMSFT
See info in area-owners.md if you want to be subscribed.

Issue Details

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Tests.Perf_UInt32

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Parse - Duration of single invocation	18.02 ns	20.91 ns	1.16	0.04	False
ParseSpan - Duration of single invocation	12.60 ns	13.90 ns	1.10	0.04	False
TryParse - Duration of single invocation	18.60 ns	20.90 ns	1.12	0.02	False
TryFormat - Duration of single invocation	10.07 ns	14.28 ns	1.42	0.02	False
Parse - Duration of single invocation	12.48 ns	14.67 ns	1.18	0.05	False
TryFormat - Duration of single invocation	6.22 ns	8.87 ns	1.43	0.08	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.Parse(value: "4294967295")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.90963580322242 > 18.784444736706895.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -12.874258442904152 (T) = (0 -20.685657621666795) / Math.Sqrt((0.3930459224307238 / (63)) + (0.12439460999108248 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.15731933657549754 = (17.873768257320886 - 20.685657621666795) / 17.873768257320886 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.ParseSpan(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.904170501323676 > 13.446475611467925.
IsChangePoint: Marked as a change because one of 4/8/2022 10:19:59 AM, 4/16/2022 5:36:25 PM, 5/30/2022 9:21:47 AM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -23.580213507568608 (T) = (0 -13.949503114739542) / Math.Sqrt((0.0714804080330334 / (61)) + (0.011735460891892903 / (5))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (61) + (5) - 2, .025) and -0.11145069804060415 = (12.550716949776868 - 13.949503114739542) / 12.550716949776868 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryParse(value: "4294967295")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.90375599341185 > 19.736056437729772.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -12.10778235242679 (T) = (0 -20.64415202822326) / Math.Sqrt((0.23215992784562625 / (63)) + (0.09534218330846458 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.12416607372545044 = (18.36396997803821 - 20.64415202822326) / 18.36396997803821 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryFormat(value: 4294967295)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.282920456766071 > 10.581393941692605.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/5/2022 12:13:02 AM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -160.85537211932368 (T) = (0 -14.246516574728334) / Math.Sqrt((0.0259529440371242 / (63)) + (0.0009960487206765827 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.44501742446885745 = (9.859062135506706 - 14.246516574728334) / 9.859062135506706 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.Parse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.667848528809003 > 13.221865998475938.
IsChangePoint: Marked as a change because one of 3/26/2022 4:52:22 AM, 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -13.61347973216338 (T) = (0 -14.532805035709623) / Math.Sqrt((0.0620460584752799 / (63)) + (0.05094260911968003 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.14358556218507731 = (12.70810468080887 - 14.532805035709623) / 12.70810468080887 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryFormat(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.874208546011742 > 6.533805810690146.
IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -37.744626161663 (T) = (0 -9.017894593080138) / Math.Sqrt((0.061713030120306656 / (63)) + (0.01775431853011536 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.5328427078851965 = (5.883118043808798 - 9.017894593080138) / 5.883118043808798 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Tests.Perf_Int64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryParseSpan - Duration of single invocation	29.28 ns	35.90 ns	1.23	0.02	False
TryParse - Duration of single invocation	28.96 ns	36.10 ns	1.25	0.02	False
TryParse - Duration of single invocation	28.67 ns	35.58 ns	1.24	0.00	False
TryParse - Duration of single invocation	13.85 ns	15.26 ns	1.10	0.02	False
ParseSpan - Duration of single invocation	29.42 ns	36.85 ns	1.25	0.01	False
ParseSpan - Duration of single invocation	29.17 ns	36.29 ns	1.24	0.01	False
Parse - Duration of single invocation	29.15 ns	35.93 ns	1.23	0.01	False
Parse - Duration of single invocation	28.73 ns	35.85 ns	1.25	0.00	False
Parse - Duration of single invocation	13.71 ns	14.99 ns	1.09	0.03	False
ParseSpan - Duration of single invocation	14.11 ns	15.13 ns	1.07	0.01	False
TryParseSpan - Duration of single invocation	29.44 ns	36.22 ns	1.23	0.00	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int64.TryParseSpan(value: "9223372036854775807")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.900716326473166 > 30.429464294572007.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -73.05297825945446 (T) = (0 -35.907973090939) / Math.Sqrt((0.5706683460185307 / (62)) + (4.6626617063356646E-05 / (3))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (62) + (3) - 2, .025) and -0.24277360223567693 = (28.893414718773123 - 35.907973090939) / 28.893414718773123 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryParse(value: "-9223372036854775808")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.09783981842236 > 30.554607480463783.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -66.8531781225283 (T) = (0 -35.9529600503081) / Math.Sqrt((0.14025398858402163 / (63)) + (0.023082790265517565 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.22730376522629492 = (29.29426362810755 - 35.9529600503081) / 29.29426362810755 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryParse(value: "9223372036854775807")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.58142235169034 > 30.234881416460922.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -83.2199470412605 (T) = (0 -35.6899605880321) / Math.Sqrt((0.07790683962682238 / (63)) + (0.015909159782171292 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.23238340922205664 = (28.960111212923117 - 35.6899605880321) / 28.960111212923117 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryParse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.261934127988493 > 14.483981674952444.
IsChangePoint: Marked as a change because one of 3/26/2022 4:52:22 AM, 3/30/2022 6:37:35 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -29.580994960848166 (T) = (0 -15.289749918526136) / Math.Sqrt((0.05031466535805899 / (63)) + (0.00537326502029719 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.1092075814195988 = (13.784390022792516 - 15.289749918526136) / 13.784390022792516 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ParseSpan(value: "-9223372036854775808")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.84727049929392 > 30.9086275873624.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -61.244756338756474 (T) = (0 -36.696407848342055) / Math.Sqrt((0.05749269576901382 / (62)) + (0.040243882518615326 / (3))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (62) + (3) - 2, .025) and -0.24979800756393505 = (29.36187097935088 - 36.696407848342055) / 29.36187097935088 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ParseSpan(value: "9223372036854775807")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.29325311558419 > 30.5373217957792.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -190.03316591378754 (T) = (0 -36.28182081910127) / Math.Sqrt((0.05515825758478877 / (62)) + (0.0017737955038416974 / (3))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (62) + (3) - 2, .025) and -0.2524429686785497 = (28.96884067893499 - 36.28182081910127) / 28.96884067893499 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.Parse(value: "-9223372036854775808")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.92793820701734 > 30.631258337420388.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -22.17634450076978 (T) = (0 -36.13059356432015) / Math.Sqrt((0.10494217907618374 / (63)) + (0.2656123200248028 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.22600500179268837 = (29.470184470282987 - 36.13059356432015) / 29.470184470282987 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.Parse(value: "9223372036854775807")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.854973534418235 > 30.160064026406275.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -48.9795648018665 (T) = (0 -35.832072835745805) / Math.Sqrt((0.1487416779463161 / (63)) + (0.04823235737302959 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.22791523998032076 = (29.181226577430596 - 35.832072835745805) / 29.181226577430596 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.Parse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.988327610326593 > 14.418160884654744.
IsChangePoint: Marked as a change because one of 3/26/2022 4:52:22 AM, 3/30/2022 6:37:35 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -25.158217370356574 (T) = (0 -15.078662593838393) / Math.Sqrt((0.03530098336008355 / (63)) + (0.0065332247519817 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.09565640768781127 = (13.762218235604754 - 15.078662593838393) / 13.762218235604754 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.ParseSpan(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.133668691470513 > 14.799796602882415.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -7.896044870754496 (T) = (0 -15.421821462857048) / Math.Sqrt((0.04045834132847233 / (62)) + (0.12558167550094462 / (3))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (62) + (3) - 2, .025) and -0.11802891738947521 = (13.793759019101222 - 15.421821462857048) / 13.793759019101222 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Int64.TryParseSpan(value: "-9223372036854775808")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.21639956941242 > 30.92121567458197.
IsChangePoint: Marked as a change because one of 4/21/2022 3:06:05 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -51.92687470020792 (T) = (0 -36.35896690761533) / Math.Sqrt((0.09227762044301657 / (62)) + (0.053350921379235965 / (3))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (62) + (3) - 2, .025) and -0.24729341424862317 = (29.150291737504432 - 36.35896690761533) / 29.150291737504432 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.CreateAddAndClear<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Dictionary - Duration of single invocation	14.38 μs	17.12 μs	1.19	0.01	False
Span - Duration of single invocation	1.09 μs	1.27 μs	1.17	0.41	False
IDictionary - Duration of single invocation	15.69 μs	18.64 μs	1.19	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<String>.Dictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.117625870364577 > 14.93440068308716.
IsChangePoint: Marked as a change because one of 3/20/2022 12:44:50 AM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -20.604820487853146 (T) = (0 -16981.715544424256) / Math.Sqrt((21146.982497953864 / (63)) + (51471.054983084076 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.19115401617742828 = (14256.52376921067 - 16981.715544424256) / 14256.52376921067 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;String&gt;.Span(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2705983781760568 > 1.171387688564955.
IsChangePoint: Marked as a change because one of 5/1/2022 10:22:11 AM, 5/7/2022 5:14:33 AM, 5/27/2022 6:33:44 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -9.40347035776082 (T) = (0 -1348.54392768287) / Math.Sqrt((5185.886876618465 / (48)) + (3293.5958691364126 / (18))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (18) - 2, .025) and -0.1350156056116734 = (1188.1280935834566 - 1348.54392768287) / 1188.1280935834566 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;String&gt;.IDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.640563153698977 > 16.3060764320747.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -148.06350576110805 (T) = (0 -18617.460411816468) / Math.Sqrt((18390.8119899223 / (63)) + (422.0202055953375 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.19819563663150466 = (15537.913711784044 - 18617.460411816468) / 15537.913711784044 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Text.Json.Tests.Perf_Booleans

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
WriteBooleans - Duration of single invocation	1.28 ms	1.41 ms	1.10	0.03	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Booleans*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Booleans.WriteBooleans(Formatted: False, SkipValidation: False)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4083138362847225 > 1.3371767802884615.
IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/20/2022 10:18:01 PM, 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -19.000157394402965 (T) = (0 -1409275.8037444036) / Math.Sqrt((2465533646.103263 / (62)) + (701569.5314954243 / (3))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (62) + (3) - 2, .025) and -0.09321824855532522 = (1289107.4637719821 - 1409275.8037444036) / 1289107.4637719821 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	9380113a892ec86d147e6e7506a10eda407a82be
Compare	842b6a87aaa6b6f25555541f541d5ee202d3972a
Diff	Diff

Regressions in System.Collections.AddGivenSize<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IDictionary - Duration of single invocation	11.43 μs	13.80 μs	1.21	0.01	False
Dictionary - Duration of single invocation	10.31 μs	12.54 μs	1.22	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<String>.IDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.798332819383258 > 12.003789028223215.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -73.01182758091328 (T) = (0 -13750.654563387116) / Math.Sqrt((19799.46109350553 / (63)) + (1767.874673691752 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.18991870092908086 = (11555.961388497124 - 13750.654563387116) / 11555.961388497124 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.AddGivenSize&lt;String&gt;.Dictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.54316092908695 > 10.837595829547027.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 5/31/2022 8:51:40 AM falls between 5/22/2022 6:37:49 PM and 5/31/2022 8:51:40 AM.
IsRegressionStdDev: Marked as regression because -47.4391453766967 (T) = (0 -12511.993589076936) / Math.Sqrt((13933.601442032314 / (63)) + (5380.782194242741 / (3))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (63) + (3) - 2, .025) and -0.20508843688281608 = (10382.635171109532 - 12511.993589076936) / 10382.635171109532 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	tannergooding
Labels:	`area-CodeGen-coreclr`, `untriaged`, `refs/heads/main`, `ubuntu 18.04`, `RunKind=micro`, `Regression`, `CoreClr`, `x64`
Milestone:	-

AndyAyersMS · 2022-06-10T20:19:20Z

Closing this as dup of #70159

performanceautofiler bot assigned tannergooding May 31, 2022

performanceautofiler bot added CoreClr untriaged New issue has not been triaged by the area owner labels May 31, 2022

ghost added the untriaged New issue has not been triaged by the area owner label May 31, 2022

tannergooding transferred this issue from dotnet/perf-autofiling-issues May 31, 2022

jeffschwMSFT added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Jun 1, 2022

AndyAyersMS mentioned this issue Jun 4, 2022

JIT: defer some flow graph reordering until after loop recognition #69878

Merged

AndyAyersMS assigned AndyAyersMS and unassigned tannergooding Jun 4, 2022

AndyAyersMS removed the untriaged New issue has not been triaged by the area owner label Jun 4, 2022

AndyAyersMS added this to the 7.0.0 milestone Jun 4, 2022

AndyAyersMS closed this as completed Jun 10, 2022

ghost locked as resolved and limited conversation to collaborators Jul 11, 2022

jeffhandley added runtime-coreclr specific to the CoreCLR runtime os-linux Linux OS (any supported distro) and removed CoreClr labels Dec 28, 2022

jeffhandley added arch-x64 and removed ubuntu 18.04 labels Dec 28, 2022

[Perf] Changes at 5/30/2022 5:18:40 PM #70019

[Perf] Changes at 5/30/2022 5:18:40 PM #70019

Comments

performanceautofiler bot commented May 31, 2022

Run Information

Regressions in System.Tests.Perf_UInt32

Repro

Payloads

Histogram

System.Tests.Perf_UInt32.Parse(value: "4294967295")

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Regressions in System.Tests.Perf_Int64

Repro

Payloads

Histogram

System.Tests.Perf_Int64.TryParseSpan(value: "9223372036854775807")

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Run Information

Regressions in System.Collections.CreateAddAndClear<String>

Repro

Payloads

Histogram

System.Collections.CreateAddAndClear<String>.Dictionary(Size: 512)

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Run Information

Regressions in System.Text.Json.Tests.Perf_Booleans

Repro

Payloads

Histogram

System.Text.Json.Tests.Perf_Booleans.WriteBooleans(Formatted: False, SkipValidation: False)

Description of detection logic

Docs

Run Information

Regressions in System.Collections.AddGivenSize<String>

Repro

Payloads

Histogram

System.Collections.AddGivenSize<String>.IDictionary(Size: 512)

Description of detection logic

Description of detection logic

Docs

performanceautofiler bot commented May 31, 2022

Run Information

Regressions in System.Tests.Perf_Int16

Repro

Payloads

Histogram

System.Tests.Perf_Int16.ToString(value: 32767)

Description of detection logic

Docs

Regressions in System.Collections.TryAddGiventSize<String>

Repro

Payloads

Histogram

System.Collections.TryAddGiventSize<String>.Dictionary(Count: 512)

Description of detection logic

Docs

Run Information

Regressions in System.Text.Json.Document.Tests.Perf_DocumentParse

Repro